| MIF | how can I force a change of password for all users without having to go through everyone | 02:05 |
|---|---|---|
| === denningsrogue5 is now known as denningsrogue | ||
| mybalzitch | sed and a for loop? | 02:06 |
| MIF | how would I do that? | 02:06 |
| === rfm_ is now known as rfm | ||
| rfm | MIF, today is your lucky day, I decided to write the sed script for you. copy /etc/shadow to /root/shadow, run "sed -i -e 's/^\([^:]*:[^:]*:\)[^:]*:/\10:/' /root/shadow", verify that it changed only the third field (password age) to 0, copy /root/shadow back over /etc/shadow, remove the /root/shadow. read "man 5 shadow" to understand. | 03:19 |
| mybalzitch | aww, it's a christmas miracle / act of kindness! | 03:20 |
| mybalzitch | :) | 03:20 |
| MIF | thank you so much rfm | 03:22 |
| MIF | I don't think you understand how much that means to me! | 03:24 |
| rfm | really I would do this with a shell or python script which would read each line, split on :, replace the 3rd field only on lines with a password | 03:26 |
| rfm | but that script would be much less of a interesting puzzle | 03:26 |
| MIF | today I found out I got hacked so I had to do this, thank you so much rfm | 03:28 |
| mybalzitch | I wouldn't trust the install anymore, or possibly even the machine if someone got root access | 03:52 |
| MIF | they did not | 03:53 |
| MIF | I have removed the user, and changed all the passwords | 03:53 |
| === denningsrogue8 is now known as denningsrogue | ||
| tomreyn | MIF: do you know it wasn't a root compromise, though? | 05:36 |
| tomreyn | just changing passwords is quite likely not enough | 05:37 |
| ren0v0 | Hi | 11:00 |
| ren0v0 | I'm running RoonServer on an Ubuntu VM. I see that they are running .exe on their systemd unit, how is that possible without Wine ? | 11:00 |
| ren0v0 | "exec "$HARDLINK" --debug --gc=sgen --server "$SCRIPT.exe" "$@"" | 11:00 |
| oerheks | i think it uses wine, perhaps wine as a snap? | 11:01 |
| ren0v0 | nope, no wine installed | 11:01 |
| ren0v0 | that's what i don't understand :D | 11:02 |
| ren0v0 | here is the running process from systemd | 11:02 |
| ren0v0 | |-14937 /opt/RoonServer/RoonMono/bin/RoonServer --gc=sgen --server RoonServer.exe | 11:02 |
| oerheks | hmm, no mention in the installer, indeed >> https://help.roonlabs.com/portal/en/kb/articles/linux-install | 11:04 |
| ren0v0 | oerheks, i started looking because they are spamming 100000s of debug logs and wanted to turn them off, and they don't "allow" you. Because they can dip in apparently and read them for themselves | 11:04 |
| ren0v0 | which of course I don't like the idea of. | 11:04 |
| ren0v0 | this was an easy fix as they pass a --debug flag to the .exe | 11:05 |
| ren0v0 | but i was very shocked to see the .exe there | 11:05 |
| oerheks | hmm curious | 11:06 |
| ren0v0 | worrying. | 11:07 |
| MIF | did you see my message? | 14:23 |
| teward | MIF: the last message from you was: I have removed the user, and changed all the passwords | 14:55 |
| teward | MIF: however, the original statement by tomreyn is still accurate: | 14:55 |
| teward | <tomreyn> MIF: do you know it wasn't a root compromise, though? just changing passwords is quite likely not enough | 14:56 |
| teward | so now you are caught up :P | 14:56 |
| BlueEagle | MIF: If the user did have elevated privileges then you should also run a rootkit checker. | 14:59 |
| MIF | no it did now | 14:59 |
| MIF | *not | 14:59 |
| MIF | and I know that for a fact | 14:59 |
| Ussat | I would not bother with the rootkit checker, they are pretty much mostly crap. Assume you are rooted, nuke/pave the system and reinstall | 15:00 |
| MIF | it was this https://yoroi.company/research/outlaw-is-back-a-new-crypto-botnet-targets-european-organizations/ | 15:00 |
| MIF | Ussat: I can't just do that | 15:00 |
| Ussat | MIF "fact"..uh huh | 15:00 |
| Ussat | You better | 15:00 |
| Ussat | because you are runing a compromised system | 15:00 |
| MIF | right now the server in offline and in my work shop and I am going through everything | 15:01 |
| Ussat | and you are 100% positive you will not miss anything ? | 15:01 |
| BlueEagle | MIF: There are exploits and backdoors that are sold on the black market, so if you've had baddies in then you're rooted and they have a way in regardless. Copy the data files over to a fresh server and fix the entry point they used. | 15:02 |
| Ussat | ^^^ | 15:02 |
| MIF | the user affected was a test user that could not go any farther then /home/user/ | 15:03 |
| BlueEagle | MIF: Also if you're running any kind of content management system (read: wordpress, phpmyadm, squirrelmail) then check all those user accounts as well. | 15:03 |
| MIF | they could not access anything but there home files | 15:03 |
| Ussat | Oh...thats a heall of an assumption | 15:03 |
| BlueEagle | MIF: chroot is breakable. | 15:04 |
| MIF | did any of you read the artical I sent? | 15:04 |
| Ussat | Yup | 15:04 |
| MIF | ok, that is what got me | 15:04 |
| Ussat | Cute blog, no idea wtf Yoroi is, dont care, youre being stupid | 15:05 |
| JanC | ren0v0: .exe is just an extension; in this case it seems like that is a .NET application (running on Mono) | 15:05 |
| JanC | (so no need for WINE obviously) | 15:06 |
| JanC | if you want to re-use that hacked system, a firmware reflash might be prudent too... | 15:10 |
| JanC | Ussat: certainly looks "professional" with that circle-follows-dot-mouse-cursor :P | 15:11 |
| Ussat | Thats what made it stand out to me | 15:12 |
| JanC | not that it says anything about their skills on security | 15:13 |
| JanC | (there are plenty of professional-looking "enterprise" companies that are mostly useless) | 15:14 |
| Ussat | There are several security blogs I follow, never herd of that one. | 15:15 |
| JanC | it's an Italian company describing itself as a “Managed Security Service Provider” | 15:18 |
| JanC | founder/CEO: https://twitter.com/Marco_Ramilli & analyst/researcher: https://twitter.com/_antoniopirozzi | 15:19 |
| MIF | I am not looking for a service I was looking for somthing about the attack | 15:24 |
| Ussat | ... | 15:39 |
| MIF | what? | 15:39 |
| Ussat | I would rely on something with a bit more reputation, but yoiu do you | 15:40 |
| MIF | I am looking around the internet | 15:40 |
| MIF | what would you recomned? | 15:40 |
| JanC | Ussat: they might be fine, just mostly concentrating on the Italian market | 15:45 |
| JanC | that said, the fact that that particular malware doesn't try to "root" the system _by default_ doesn't mean it can't be used with other payloads | 15:46 |
| JanC | and because of that one should consider the system compromised | 15:47 |
| jancoow | Hi. How can I remove the disk usage from message of the day when connecting through ssh | 16:09 |
| jancoow | It takes terrible long because it needs to spin up all my disks | 16:10 |
| jancoow | nvm found it, it was in /etc/update-motd.d/50-landscape-sysinfo | 16:18 |
| a4ginatl | I am new to Unbuntu so have a bunch of dumb questions. I am trying to get Eclipse installed on Ubuntu server. Is this possible or not? I am getting the following error: An error occurred while automatically activating bundle org.eclipse.oomph.setup.installer (135). What am I doing wrong? | 17:19 |
| devster31 | logically speaking, should systemd services which run long backup tasks be considered Type=simple or Type=oneshot? | 17:21 |
| tomreyn | the wqy i understand oneshot tasks is that they are meant to be started once, but not meant to be kept running / no action needs to be taken when the process vanishes. | 17:26 |
| tomreyn | s/oneshot tasks/services of Type=oneshot/ | 17:27 |
| tomreyn | ^ devster31 | 17:29 |
| JanC | I thought Eclipse was a desktop application... | 17:29 |
| tomreyn | a4ginatl: Isn't eclipse a GUI application (servers usually don't have that)? Generally, don't expect to get support *here* with software not packaged in Ubuntu (but see which support forums those softwares provide themselves). | 17:29 |
| tomreyn | actually, eclipse is packaged in ubuntu LTS releases until and including 18.04 LTS, if just in universe. but if you'll install it you'll probably want to use current eclipse project provided packages instead. the point about desktop remains. | 17:31 |
| JanC | there still seem to be lost of eclipse-related packages in Ubuntu (but I guess they can be used for other purposes too?) | 17:33 |
| JanC | *lots of* | 17:33 |
| devster31 | tomreyn I considered oneshot as I can use execstartpost for notification commands, but it doesn't seem to be made for long-running processes | 17:35 |
| tomreyn | the "eclipse" packages in Ubuntu are *really* old though https://en.wikipedia.org/wiki/Eclipse_(software)#Releases | 17:36 |
| tomreyn | yes, i think oneshot is meant to be forking and be done, soon, or at least not managed by systemd | 17:37 |
| JanC | IIRC maintainers always complained that eclipse was a near impossible mess to package properly | 17:39 |
| tomreyn | a very first for java software, i'm sure :) | 17:41 |
| JanC | yeah, it used to be that every FOSDEM edition had at least one talk complaining about that :) | 17:42 |
| a4ginatl | Thanks, so what you are telling me is to install eclipse on a client system and use it that way? | 18:28 |
| MIF | is rsync installed by default? | 19:39 |
| tieinv | MIF i believe it is , i just ssh'd into my server 20.04.1 and rsync was there | 19:41 |
| MIF | okk | 19:41 |
| MIF | what are some good ways to harden up a server? | 19:42 |
| tieinv | get rid of passwords and use ssh | 19:42 |
| MIF | ok | 19:42 |
| MIF | what else? | 19:43 |
| tieinv | changet the default ssh port and disable root login | 19:48 |
| MIF | ok | 19:48 |
| MIF | I have done that | 19:48 |
| mybalzitch | changing default ssh port is at best security through obscurity. you shouldn't be allowing password based authentication via ssh anyway | 19:51 |
| MIF | ok, when I put my server back online the first thing i will do is get the ssh keys then lock the password auth | 19:53 |
| MIF | what else do I need to know? | 19:54 |
| JanC | might be useful to block IPs that try to connect too often too frequently to the SSH port(s) also | 20:07 |
| MIF | I have fail2ban | 20:09 |
| JanC | also, if possible, implement all the restrictions _before_ you put the server back online :) | 20:13 |
| MIF | will do | 20:14 |
| MIF | because I bet that my server as soon as it comes online will start getting attacked again | 20:14 |
| MIF | would you say that? | 20:15 |
| Ussat | MIF, you are asking about ways to harden a server, but earlkier you were 100% positive you were not rooted | 20:21 |
| Ussat | This does not inspire confidence | 20:21 |
| MIF | I still am 100% sure the root account was not hacked, I am just looking for ways to make sure I don't get hacked again | 20:22 |
| MIF | is there any other ideas/tips/anything | 20:35 |
| teward | what you're asking is how to harden a server, but you were 100% positive you were not rooted. These seem like... conflicting viewpoints | 20:37 |
| teward | if you're sure you weren't rooted, then you should have already hardened your server before hand | 20:37 |
| teward | if you aren't sure if you've been rooted and are asking how to HARDEN your system | 20:37 |
| teward | you're in the wrong line of thought, start your server over. | 20:37 |
| teward | THEN harden it | 20:37 |
| teward | THEN put it online | 20:37 |
| teward | and as for 'getting attacked' a lot of the activity on the Internet is Bad. | 20:38 |
| teward | if you're not fully invested into taking the time to actively monitor/protect your systems actively (it's a full time job!) then you probably shouldn't be *running* a server to start. | 20:38 |
| teward | hate to be completely blunt but... :P | 20:38 |
| teward | (I'm in the IT SEcurity field, it's always a lot of work to actively protect even just ONE server from 'all threats') | 20:38 |
| teward | MIF: disallow Password authentication, use SSH keys only. Disallow root login directly over SSH. Enable `ufw` firewall and properly configure it to ONLY permit connections you want to permit. WHITELIST what IPs can connect to the SSH port, block all other SSH attempts with the firewall. Implement Fail2ban for excessive connections / failed lookups / etc. on all your specific services, where you can access the logs, etc. for those for | 20:40 |
| teward | fail2ban to view. | 20:40 |
| teward | IDEALLY: | 20:40 |
| teward | put your system behind some type of firewall that has threat intelligence datasets into the 'blocklists' so that connections from known badness are blocked by default | 20:40 |
| MIF | ok, what I know for a fact, 1. the user did not have root. 2. they started on Dec 14, 2020 at 20:13:25 CST. 3. They used my server to attack others. 4. Fail to ban blocked over 500 ips in that 10 day peorid | 20:40 |
| teward | here's a question: | 20:41 |
| teward | this is a 'new rogue user' yes? | 20:41 |
| teward | how'd that user get *created* to begin with? | 20:41 |
| MIF | I created the user as a test user mounths before my server went online | 20:41 |
| teward | don't create users that you don't intend to use / harden | 20:42 |
| teward | that's another case of Admin Failure here. | 20:42 |
| teward | did this 'test user' ever have `sudo` access in the time since it was 'created'? | 20:42 |
| MIF | no | 20:42 |
| teward | > used my server to attack others | 20:42 |
| teward | attack how? | 20:42 |
| teward | (important thing you're missing) | 20:42 |
| teward | > fail2ban blocked over 500 IPs in that 10 day period | 20:42 |
| teward | means jack squat | 20:42 |
| teward | i have f2b lists that're over 2000 **per day** | 20:43 |
| MIF | You are in the IT SEcurity field, I am in high school | 20:43 |
| teward | and that's just what gets past the 2mil+ threats that're blocked by the IDS/IPS on the border thanks to threat intel :P | 20:43 |
| MIF | I know | 20:44 |
| MIF | I talked to my ISP | 20:44 |
| dax | MIF: every ssh server on port 22 on the public internet gets hundreds/thousands of ssh login attempts per day, because botnets | 20:44 |
| MIF | I found that out | 20:44 |
| dax | would be interested in the answer to teward's "attack how?", tbh | 20:45 |
| teward | ^ this | 20:45 |
| teward | because I'm real curious how your server was 'attacking' others - it'll help to ID exactly what chaos was going on | 20:46 |
| teward | hacked or otherwise | 20:46 |
| teward | IT Security Professional or High School Student, doesn't matter, if you were told your system was attacking others you'll have information about the 'attack' itself | 20:46 |
| teward | likely from yoru ISP :P | 20:46 |
| teward | or whoever reported the server was being used maliciously | 20:46 |
| teward | (alternatively PM me your IP i'll run it through greynoise and that should give me an idea of the type of attacks your system was doing) | 20:46 |
| MIF | ok | 20:47 |
| JanC | teward: from what he linked to it seemed like his server became part of a botnet, so really it could have been used for almost anything... | 20:48 |
| JanC | including trying to infect others :) | 20:48 |
| MIF | that is what I think was going on | 20:49 |
| JanC | or being part of some DDoS | 20:49 |
| teward | JanC: then I'd start with the "Nuke Your System" approach | 20:51 |
| JanC | obviously | 20:51 |
| teward | because even if it was running as a single user botnets themselves tend to take over the machine overall | 20:51 |
| teward | MIF: if you haven't started by nuking the system and starting 'fresh' and restoring from last known good backups you're starting off bad | 20:51 |
| MIF | you are going to hate me for this | 20:51 |
| JanC | botnets often work as a "simple" user | 20:51 |
| MIF | but I have not made backups yet | 20:52 |
| teward | JanC: depends on the infection methond | 20:52 |
| teward | if it was a simple "run slave agent of botnet as user" then yes | 20:52 |
| JanC | but if they _can_ get root they won't refuse that opportunity, of course | 20:52 |
| teward | if it was that leading to systemic infection of multiple bot agents, etc. then it's possible | 20:52 |
| teward | given that OP was 'breached' with password authentication and /etc/passwd is visible I wouldn't be surprised if they did dictionary attacks on ${ALL_ACCOUNTS} on system | 20:53 |
| teward | to try and break in | 20:53 |
| teward | hence the 'start over' statement | 20:53 |
| JanC | for example | 20:53 |
| MIF | the user did not have a password | 20:53 |
| JanC | so then how did they get in through SSH? | 20:53 |
| teward | ^ that | 20:53 |
| teward | default disallows passwordless ssh without SSH keys | 20:54 |
| teward | "PermitEmptyPasswords no" is a default in sshd_config in all Ubuntu deployments of ssh server | 20:54 |
| teward | from install of openssh-server | 20:54 |
| MIF | that is a good question, I never changed any of the sshd conifg settings | 20:54 |
| MIF | and I put my IP in greyNoise | 20:55 |
| teward | should show you SOMETHING if you are using the visualizer - at least if you were part of a botnet heh | 20:56 |
| teward | in either case | 20:56 |
| teward | you've got 'basic' hardening steps i've identified | 20:56 |
| JanC | no matter what, you want to do a re-install :) | 20:56 |
| MIF | ok | 20:56 |
| teward | but if the user had no password AND they got in via SSH AND you left the sshd_config the same as install then your hacker infiltrated some other way than SSH | 20:56 |
| JanC | after saving data | 20:56 |
| teward | and I would say that you should ASSUME your system was rooted because you don't know how they got in | 20:56 |
| teward | and torch it after saving any data you need to keeo | 20:57 |
| teward | (then reinstall scratch) | 20:57 |
| teward | from scratch* | 20:57 |
| MIF | yea, I don't have a good way to back that up | 20:58 |
| MIF | I might of set "PermitEmptyPasswords on" I don't know for sure | 21:13 |
| Ussat | WAT !!!! you dont have backups ??? | 21:15 |
| MIF | I said you would not like it | 21:15 |
| MIF | I don't have any good place to store them | 21:16 |
| Ussat | Dude.... | 21:16 |
| Ussat | and "I might of set "PermitEmptyPasswords on" I don't know fo" is, um..scary | 21:17 |
| MIF | this server has been running for mounths offline | 21:17 |
| MIF | (did not have a public IP) | 21:17 |
| Ussat | with no backups | 21:17 |
| Ussat | and in prod | 21:18 |
| RoyK | MIF: didn't have a public ip and then the ISP introduces ipv6 :D | 21:18 |
| MIF | no I got my dad to pay for the ipv4 | 21:18 |
| Ussat | ... | 21:18 |
| MIF | I just ran the 500+ IPs from my fail2ban list | 21:18 |
| MIF | 571 came up as 571 | 21:19 |
| Ussat | WHat was this server doing ? | 21:19 |
| MIF | 571 came up as Malicious | 21:19 |
| JanC | might be useful for some people in here too: https://www.humblebundle.com/books/linux-apress-books | 21:50 |
| jessequinn | hi, quick question. i tried to enable firewalld but i am getting the following error: `Error: COMMAND_FAILED: 'python-nftables' failed: internal:0:0-0: Error: Could not process rule: No such file or directory` when im doing a firewall-cmd --reload. any ideas? | 22:33 |
| tomreyn | sounds like the rule directory doe snot exist or was not accessible. see journalctl for potential related errors (audit/apparmor?), try to find out where the rules should be located and check whether this directory eixsts and appears to be accessible to the service. | 22:41 |
| tomreyn | related (yes, redhat, may or may not be relevant)? https://bugzilla.redhat.com/show_bug.cgi?id=1836571 https://bugzilla.redhat.com/show_bug.cgi?id=1817205 | 22:43 |
| ubot3 | bugzilla.redhat.com bug 1836571 in Fedora "firewalld startup failure: COMMAND_FAILED: 'python-nftables' failed: internal:0:0-0: Error: Could not process rule: File exists" [Unspecified, New] | 22:43 |
| ubot3 | bugzilla.redhat.com bug 1817205 in Red Hat Enterprise Linux 8 "firewalld rules broken by nftables service if started later" [High, Closed: Errata] | 22:44 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!