CPU and traffic abnormal behavior on VPS!

 · 3 mins read

Recently I found my VPS worked extremely unusual theses days ’cause I continuously received serval alert emails from LINODE TEAM. They are like these:

CPU

traffic

it ran out of more than 600G traffic with in 20 days! Can you believe that?

IO

I’ve tried different kinds of Linux network monitoring tools, like iptraf, nethogs, nethogs…

Now I am working on it, I hope I can find the chief culprit!

To be continued…

(2017/12/24)


2017/12/29

have been busy these days, today I got a time to tackle this finally…

Luckily, I found the nasty thing running on the VPS and how to thoroughly remove this!

I use top to check the status of processes on my server, find the wipefs take up too much cpu usage all the time.

First of all , KILL it!

Then I search it online, it seems that many people have encountered this problem and have no idea what it is. In fact, wipefs is a built-in tool to wipe files, which path should be /usr/bin/wipefs. But the wipefs on my server is pointed at /bin/wipefs (use: ps aux)

Even if you kill it , it would start some other time automactically.

It is said that it’s a mining program intentionally injected into yours system by someone. In other words, they are take advantage of you to make their own money! It’s really pissing and intolerable!

WHAT THE THING DOES:

  1. do mining, take up massive cpu
  2. copy itself to /bin/wipefs, create service /etc/init.d/wipefs, create links in /etc/rc(x).d and /etc/rc.d/rc(x).d to start up with system.
  3. release subprocess to /bin/ddus-uidgen, create service /etc/init.d/acpidtd and its links in *\** /etc/rc(x).d **** and ***/etc/rc.d/rc(x).d ***
  4. modify *\**/etc/resolv.conf ***, maybe resolve ip for mining server
  5. modify /etc/crontab, create timed task for it’s job

WHAT WE SHOULD DO:

  1. delete timed task in /etc/crontab
  2. delete these files below:

    /bin/wipefs

    /etc/init.d/wipefs

    /bin/ddus-uidgen

    /etc/init.d/acpidtd

    /etc/rc*/S01wipefs

    /etc/rc*/acpidtd

  3. overhaul the system, check upon vulnerabilities, pay attention to ssh privileges, firewall etc…

SCREENSHOTS FROM MY TERMINAL:

except wipfs, there is a httpsd, I don’t know what it is, just remove!

crontab


you can find out all the relating files use find command then delete


you might see this werid thing when you try to delete this file:

you may get confused because, like I, loggin as root but still can’t do that, why?

here it is! The program use chatter to protect it from deleting, how smart!

In order to do that, you should follow these steps:

root@localhost:/bin# lsattr ddus-uidgen
----i--------e-- ddus-uidgen
root@localhost:/bin# chattr -i ddus-uidgen
root@localhost:/bin# lsattr ddus-uidgen
-------------e-- ddus-uidgen
root@localhost:/bin# rm -f ddus-uidgen
root@localhost:/bin#

Congratulations!

All done!


about SSH:

some said using credentials rather than password to loggin ssh is better. Because first of all, it leaves out password which is fairly convenient, and also you can close the way of connect with password to reinforce the security of your system, protecting it from someone using brute force. Follow these:

  1. ssh-keygen -t rsa
  2. copy your id_rsa.pub to server, in the .ssh directory of the specific loggin user. Rename it to authorized_keys.
  3. chmod 700 ~/.ssh

    chmod 644 ~/.ssh/authorized_keys

PS: If the privileges of ~/.ssh and authorized_keys is not the exactly same above, it may not work properly.