Author Topic: Advice Needed: Offboarding Sysadmin  (Read 396 times)

0 Members and 1 Guest are viewing this topic.

Offline The Latinist

  • Cyber Greasemonkey
  • Technical Administrator
  • Too Much Spare Time
  • *****
  • Posts: 7625
Advice Needed: Offboarding Sysadmin
« on: March 08, 2019, 05:49:25 PM »
My employer has just parted ways with our contracted sysadmin (who was the husband of an employee who just resigned after having her contract non-renewed).  I’m the in-house guy most familiar with the system and have been managing things day-to-day with advice and assistance from this contractor.  Now I find myself in the position of having to secure a system completely designed by this individual, who has had access to every root password on every device and bit of infrastructure we have.  I have already disabled VPN access and deleted his credentials on our domain.  I realize that I will have to change dozens off passwords and do my best to ensure there are no dummy accounts, etc.

I also realize that the best way to handle this would have been to prepare in advance with appropriate policies and procedures.  Indeed, I have been working toward establishing such procedures for months now.  If I had been consulted, I would have strongly recommended a smoother transition with every effort made to part on the best of terms.

To be clear, I don’t suspect this person of anything and have no reason to doubt his honesty.  But it’s my job now that hes gone to anticipate and prepare against the worst, especially until we can hire a new consulting contractor.

I know some of you have IT expertise, and I would appreciate any input you might have on things I shouldn’t overlook, etc.
I would like to propose...that...it is undesirable to believe in a proposition when there is no ground whatever for supposing it true. — Bertrand Russell

Online Desert Fox

  • Poster of Extraordinary Magnitude
  • **********
  • Posts: 19101
  • Hopeful Non-Theist
    • Kitsune's Web Page
Re: Advice Needed: Offboarding Sysadmin
« Reply #1 on: March 08, 2019, 06:40:29 PM »
My roommate was a cyber security specialist - Can I send her it?
"Give me the storm and tempest of thought and action, rather than the dead calm of ignorance and faith. Banish me from Eden when you will; but first let me eat of the fruit of the tree of knowledge."
— Robert G. Ingersoll

Offline The Latinist

  • Cyber Greasemonkey
  • Technical Administrator
  • Too Much Spare Time
  • *****
  • Posts: 7625
Re: Advice Needed: Offboarding Sysadmin
« Reply #2 on: March 08, 2019, 06:46:38 PM »
My roommate was a cyber security specialist - Can I send her it?


I’m happy to get any advice I can, TBH.  This is not my area of expertise and I had hoped to get some serious training in the area over the summer before I had need of such skills.
I would like to propose...that...it is undesirable to believe in a proposition when there is no ground whatever for supposing it true. — Bertrand Russell

Online Desert Fox

  • Poster of Extraordinary Magnitude
  • **********
  • Posts: 19101
  • Hopeful Non-Theist
    • Kitsune's Web Page
Re: Advice Needed: Offboarding Sysadmin
« Reply #3 on: March 08, 2019, 07:31:31 PM »
I sent it to her.
"Give me the storm and tempest of thought and action, rather than the dead calm of ignorance and faith. Banish me from Eden when you will; but first let me eat of the fruit of the tree of knowledge."
— Robert G. Ingersoll

Offline moj

  • beer snob
  • Poster of Extraordinary Magnitude
  • **********
  • Posts: 10050
Re: Advice Needed: Offboarding Sysadmin
« Reply #4 on: March 11, 2019, 09:32:41 AM »
Do an audit of all local accounts on servers and service accounts on all devices. Make a spread sheet or something to keep track of them. A lot of times admins will create there own local admin account on a server or use a network or service account with elevated permissions so it doesn't track them individually on log in. Once you know how  many accounts you have than can come up with a schedule for updating the passwords. Beware on the service accounts, a lot of applications that use service accounts may have places the passwords are embed so if you change the password in Active directory will also have to change it in the application. This may not be an issue but something to be aware of before you make changes in AD.

Offline mobyfubar

  • Brand New
  • Posts: 2
  • Bryan
Re: Advice Needed: Offboarding Sysadmin
« Reply #5 on: March 11, 2019, 09:59:58 AM »
If your systems support ssh (mostly this is on Linux), in addition to changing account passwords, check whether there are ssh keys authorized to login. This is in ~/.ssh/authorized_keys. Especially check under /root on all hosts.
==BD

Offline The Latinist

  • Cyber Greasemonkey
  • Technical Administrator
  • Too Much Spare Time
  • *****
  • Posts: 7625
Re: Advice Needed: Offboarding Sysadmin
« Reply #6 on: March 11, 2019, 12:21:09 PM »
Do an audit of all local accounts on servers and service accounts on all devices. Make a spread sheet or something to keep track of them. A lot of times admins will create there own local admin account on a server or use a network or service account with elevated permissions so it doesn't track them individually on log in. Once you know how  many accounts you have than can come up with a schedule for updating the passwords. Beware on the service accounts, a lot of applications that use service accounts may have places the passwords are embed so if you change the password in Active directory will also have to change it in the application. This may not be an issue but something to be aware of before you make changes in AD.

Thank you for this suggestion. There are a few service accounts, which I've now added to my list of things to change once I've figured out exactly where each is used.  After deleting his account I discovered that he had set up some backup jobs using his own credentials, which caused a bit of a headache.  I've created a new service account for that purpose.

If your systems support ssh (mostly this is on Linux), in addition to changing account passwords, check whether there are ssh keys authorized to login. This is in ~/.ssh/authorized_keys. Especially check under /root on all hosts.

Thanks for this suggestion.  SSH is disabled on all of our servers, but our switches are managed through SSH.  Checking them, there don't seem to be any stored client certificates.  The switches were all managed from a root account with a password.  I've changed those passwords and intend to set up NPS logins so that nobody needs the root passwords.

During this process I'm taking the opportunity to replace the root passwords on everything with unique, random passwords (well-documented).  Previous guy had the root passwords on pretty much every piece of hardware set the same.
I would like to propose...that...it is undesirable to believe in a proposition when there is no ground whatever for supposing it true. — Bertrand Russell

Offline Soldier of FORTRAN

  • Reef Tank Owner
  • *********
  • Posts: 9339
  • Cache rules everything around me.
Re: Advice Needed: Offboarding Sysadmin
« Reply #7 on: March 11, 2019, 01:46:03 PM »
Are things being deleted that remove logs, etc. with them?  I'm just some guy taking classes but 'disable rather than delete' came up in that context.  If a problem's discovered six months from now, you don't want the investigation to hit a wall because something was deleted.
If global warming is real then how come I just felt this chill down my spine?

 

personate-rain