Rendered at 03:01:02 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
chasil 9 hours ago [-]
I have a few observations about this article.
Generally, try not to use SCP. It has been a crufty old program from the Berkeley R-Utilities, but newer OpenSSH releases have rewritten it to use the sftp-server server instead. There will be wildly different behavior between these implementations.
If you need something that SFTP cannot do, then use tar on both sides.
PuTTY has implemented their pscp to prefer the sftp-server for many years, in a long prediction of the eventual abandonment. Their pscp implementation is a better drop-in replacement than the OpenSSH solutions.
The allure of SCP is retry on failure, which is somewhat more difficult with SFTP:
until scp source.txt user@target:dir/
do echo target down; sleep 300
done
Converting that to pscp is much easier than SFTP.
I also have an older rhel5 system where I am running tinysshd to use better SSH crypto. Due to upgrades, NFS is now squashing everything to nobody, so I had to disable precisely these checks to let users login with their authorized_keys. I can post the code if anybody is curious.
gerdesj 1 hours ago [-]
"I have a few observations about this article."
I have a few observations about this comment.
Generally use whatever works to do the job. Do think about security, so if you end up streaming stuff across the internet using scp really consider your life choices.
In reality, you will probably be copying stuff on or across local nets or across a VPN because port 22 is (of course) unavailable from !RFC1918(etc).
Use the tool for the job and don't pontificate (unless you know best!)
chasil 12 minutes ago [-]
Allow me to introduce you to the Berkeley R-Utilities:
I suggest that you use them for all your production needs, exclusively!
beepbooptheory 22 minutes ago [-]
I for one appreciate this genre of comment. It's these days kinda one of the only things still redeemable on this particular website. I dont have to agree with it or end up using it, but its always good to see.
You get rid of some (wannabe-)greybeard nerd sniping someone's technical blog about an obscure linux util? What would even be left here? Just B2B SaaS AI blah blah forever?
You wan't them to couch this in "great job! Use whatever your comfortable with!" Are we really at that point?
elevation 7 hours ago [-]
I occasionally use `scp` around my network and have for years. It works great and its simple interface is easy to remember. I don't want to sftp if I have to use tar on both sides. I might type rsync and but then I remember something about the trailing slash will cause the command to behave differently the second time. I just don't need yet another syntax I'll misremember. As long as scp is in my distro's repositories, I'll be using it.
fsckboy 5 hours ago [-]
easy to remember, if you don't use trailing slashes ever, it will just work every time
4 hours ago [-]
0ckpuppet 3 hours ago [-]
rsync -avz -e ssh /local/ftw/ user@foo:/ftw/
eikenberry 6 hours ago [-]
> Their pscp implementation is a better drop-in replacement than the OpenSSH solutions.
What makes it a better drop in replacement?
chasil 5 hours ago [-]
Several reasons.
-PuTTY pscp allows raw passwords on the command line, or from a file. OpenSSH is unreasonable in refusing to do this.
-Scripting can adapt to a .netrc easily; OpenSSH will never do this.
-Modern OpenSSH is a nightmare when using legacy crypto, while pscp is fluid. There is nothing wrong with hmac-md5, and no reason to refuse it. I will take PuTTY or dropbear in a heartbeat over these burned bridges and workarounds.
-pscp does not link to dozens of libraries as ssh/scp does, so it is easier to build with less dependency. The ldd output of ssh and scp on rhel9 is 23 libraries, while PuTTY is 3 [package obtained from EPEL].
-pscp strongly leans to SFTP on the backend and can be directed to use it exclusively, so there is no ambiguity.
-Using pscp with a retry on fail is much easier than sftp -b.
-The wacky cipher control on rhel8 does not impact the PuTTY tools.
That is an extensive list.
jolmg 4 hours ago [-]
> -PuTTY pscp allows raw passwords on the command line, or from a file. OpenSSH is unreasonable in refusing to do this.
You can use `sshpass` to force it through a command line argument. However, arguments can be viewed by any process through `/proc`, `ps`, etc. It's pretty reasonable to not support exposure of the password like that, especially since you can force it through using another tool if you really, really need to.
chasil 2 hours ago [-]
Both pscp and psftp have -pwfile.
It is not reasonable to insist on keys for batch use.
Not at all.
tialaramex 2 hours ago [-]
It's completely crazy to use passwords when you needn't. Passwords are a human readable shared secret, they were already obsolete when SSHv1 was invented last century.
From the outset SecSH (SSHv2, the thing you actually use today and if you're younger, likely the only thing you ever have used) has public key authentication as a Mandatory To Implement feature. Implementations where that doesn't work aren't even SSH, they're garbage.
chasil 3 minutes ago [-]
I am forced by external vendors and internal security to use password authentication for SFTP.
I do not have a choice!
This grew out of FTP less than a decade ago. Everyone has always known password auth; it cannot die.
Are you on the same planet as the rest of us?
extraduder_ire 6 hours ago [-]
>If you need something that SFTP cannot do, then use tar on both sides.
Wouldn't tar do the exact same thing to that file's permissions?
chasil 5 hours ago [-]
Likely, but maintaining hard links is more of what I was thinking.
jolmg 5 hours ago [-]
SCP protocol is fine and convenient as long as people understand that the remote file arguments are server-side shell code, and the consequences that implies.
You get the benefit of being able to e.g. get your last download off your desktop to your laptop like this:
scp -TO desktop:'downloads/*(oc[1])' .
or this if you're on bash:
scp -TO desktop:'$(ls -t downloads/* | head -1)' .
or pull a file from a very nested project dir for which you have setup dynamic directories (or shell variables if you're on bash):
Just don't pull files from an SCP server that may be malicious. Use on trusted servers. If you do the following on your home dir:
scp -TOr malicious:foo/ .
That may overwrite .ssh/authorized_keys, .zshrc, etc. because `foo/` is server-side shell code. The client can't say that `.zshrc` resulting from the evaluation of `foo/` doesn't make sense, because it might in the remote shell language.
> If you need something that SFTP cannot do, then use tar on both sides.
No reason to make things inconvenient between personal, trusted computers, just because there may be malicious servers out there where one has no reason to SCP.
Something else to note is that your suggestion of using `tar` like `ssh malicious 'tar c foo/' | tar x` faces basically the exact same problem. The server can be malicious and return .ssh/authorized_keys, .zshrc, etc. in the archive for `tar x` to overwrite locally basically exactly the same way. This goes with the point of this SE answer:
> I'd say a lot of Unix commands become unsafe if you consider a MITM on SSH possible. A malicious sudo could steal your password, a malicious communication client could read your mails/instant messages, etc. Saying that replacing scp with sftp when talking to a compromised server will somehow rectify the situation is very optimistic to say the least. [...] In short, if you don't pay attention to which servers you SSH into, there's a high risk for you to be screwed no matter which tools you use, and using sftp instead of scp will be only marginally safer. --- https://unix.stackexchange.com/questions/571293/is-scp-unsaf...
I think this whole problem with SCP just stems from not having properly documented this aspect in the manpage, so people expected it to just take filepaths.
Onavo 4 hours ago [-]
What I want is to be able to drag and drop files in my remote server to and from my desktop as if it's an NFS/NAS. What's the best option for this that will fully saturate the link?
biglost 2 hours ago [-]
I don't know about saturating the link, Buy sshfs can mount remote dirs and then you can drag and drop
CIFS/Samba has had a lot of effort put into it, so setup Tailscale and just use that, if you're after drag and drop.
mistrial9 8 hours ago [-]
you sound so wise and produce excellent reference, but in the next breath you show NFS in use?
signed -confused
eikenberry 6 hours ago [-]
What would you use for remote mounting filesystems? I don't know of any that are simply superior (w/o caveats/tradeoffs).
gchamonlive 7 hours ago [-]
Why is it so self-evident that NFS is bad?
chasil 5 hours ago [-]
I upvoted you, and yes, cleartext NFS is a concern.
I had it wrapped in stunnel TLS, but I ripped that out recently as I am retiring and the new staff is simply not capable of maintaining that configuration.
My users were yelling, and the patch to tinysshd to omit all permissions checks silenced the complaints. No, it's not pretty.
nubinetwork 4 hours ago [-]
While it wouldn't prevent the issue they described, I prefer to pull, rather than push. My thinking is, if you pull, you're still connected. If you push, as soon as the push finishes, you're locked out.
3 hours ago [-]
procaryote 9 hours ago [-]
This is a useful tip!
but also... who has a dir with 777 permissions? Is that something people do nowadays?
SoftTalker 6 hours ago [-]
I've seen users who have every file set to 777. They do it to "avoid permissions issues"
Retr0id 2 hours ago [-]
Heh I can remember doing that, in the distant past.
easterncalculus 8 hours ago [-]
My guess would be mounting an NTFS partition - with ntfs-3g it will load everything as 777 just by default, since it can’t translate the permissions.
chasil 9 hours ago [-]
Well, everybody has 1777 as /tmp (with the sticky bit).
I assume using `./*` rather than `.` in the `scp` command would have worked around the issue?
hrmtst93837 6 hours ago [-]
Using './*' would have avoided this in most shells because ordinary globbing excludes dotfiles, so .ssh and authorized_keys are not matched. In my experience scp is brittle for bulk syncs, so I run rsync -a --exclude='.ssh' --dry-run ./ user@host:~/target to verify before I commit the changes. I keep an out of band recovery path, like a temporary deploy key, a nonprivileged rescue user, or console access, as the only reliable way to avoid being locked out at 3AM.
Biganon 6 hours ago [-]
The problem was not scp'ing the .ssh/ directory. The problem was scp'ing a directory whose permissions were 777, and "mapping" it (cannot find a better term) to a remote directory, which happened to be the home directory. The remote home directory therefore had its permissions changed to 777, which was deemed "too open" by openssh which refuses to use any file in it.
malicka 9 hours ago [-]
Yes, since it would’ve copied the globbed files, rather than the current directory itself.
tracker1 8 hours ago [-]
I accidentally nuked my hosted server's network stack with a config error... my bigger mistake was generating a massive random password for the root account... the remote terminal management console didn't support pasting and the default config only gave you like 30s to login.... not fun at all.
Script all the things. double-check your scripts... always be backing up.
jonathanlydall 7 hours ago [-]
> the remote terminal management console didn't support pasting and the default config only gave you like 30s to login
I would have used AutoHotkey or something similar in such a scenario.
gchamonlive 7 hours ago [-]
Also a gentle reminder that backups without periodic drills are just binary blobs. I had an instance where for some reason my Borg backups where corrupted. Only caught them with periodic drills.
sowbug 9 hours ago [-]
Related: In my Bash logout script I have a chmod that fixes authorized_keys. It won't help with scp because that's non-interactive, but it has helped the other 999 times I've forgotten to clean up the mess I made during an ssh session.
jamiesonbecker 4 hours ago [-]
Classic OpenSSH safety check: if /home/$user (or ~/.ssh) is too open, or ownership/modes are off, sshd will refuse pubkey auth. Annoying, but correct.
If you still have some access (console, password login, another sudo user), this usually fixes it:
Not to shill too hard, but this exact "keys/perms/sudo drift" failure mode is why Userify exists (est. 2011): local accounts on every box + a tiny outbound-only agent that polls and overwrites desired state (keys, perms, sudo role). If scp/rsync/deploy steps clobber stuff, the next poll re-converges it (cloud default ~90s; self-host default ~10s; configurable). Removing a user also kills their sessions. No inbound ports to nodes, no PAM/NSS hooks, auditable.
Ah, file permissions. My old friend. Good thing this happened on a 'local' server and not a remote VPS.
LoganDark 7 hours ago [-]
You did not transfer the files within a directory. You transferred the directory itself, via `.`. That is why scp changed the permissions of your home directory itself; if you instead had transferred via `*` I am sure you would not have had this problem.
MomsAVoxell 7 hours ago [-]
Done stupid stuff like this enough times that I just use tar, and also make a sandbox directory to receive it, to double-check whats going to happen, before un—tar’ing it again into the destination intended and/or do a manual move.
Too many burned fingers to not do this little dance almost every other time.
Actually, I lied, I just use rsync like an insane person.
0ckpuppet 3 hours ago [-]
rsync over ssh
crest 10 hours ago [-]
It's nice to see people sharing their mistakes too.
TZubiri 8 hours ago [-]
Getting locked out of a server must be a cannonical experienc in the sysadmin journey, like checking the logs to see you are being attacked as soon as your online, or trying to build your own linux from scratch without bloat.
roelschroeven 10 hours ago [-]
tl;dr: I you scp -r to your homedir, expect scp to copy not just files and directories but their permissions as well (which I think isn't all that surprising).
ranger_danger 10 hours ago [-]
It's not supposed to do that unless it's newly creating the destination, or you supplied the -p flag to preserve permissions... that's what the entire issue is about; it's a bug that was fixed in 10.3.
Calzifer 8 hours ago [-]
I wouldn't even expect it on newly created stuff without the -p flag. Normal cp doesn't do it.
rhier1 8 hours ago [-]
[dead]
binaryturtle 11 hours ago [-]
When I load the site in my (slightly older) Firefox I just get some random junk and gibberish (markov chain generated nonsense?)
Generally, try not to use SCP. It has been a crufty old program from the Berkeley R-Utilities, but newer OpenSSH releases have rewritten it to use the sftp-server server instead. There will be wildly different behavior between these implementations.
The backend SCP changes are documented here:
https://lwn.net/Articles/835962/
If you need something that SFTP cannot do, then use tar on both sides.
PuTTY has implemented their pscp to prefer the sftp-server for many years, in a long prediction of the eventual abandonment. Their pscp implementation is a better drop-in replacement than the OpenSSH solutions.
The allure of SCP is retry on failure, which is somewhat more difficult with SFTP:
Converting that to pscp is much easier than SFTP.I also have an older rhel5 system where I am running tinysshd to use better SSH crypto. Due to upgrades, NFS is now squashing everything to nobody, so I had to disable precisely these checks to let users login with their authorized_keys. I can post the code if anybody is curious.
I have a few observations about this comment.
Generally use whatever works to do the job. Do think about security, so if you end up streaming stuff across the internet using scp really consider your life choices.
In reality, you will probably be copying stuff on or across local nets or across a VPN because port 22 is (of course) unavailable from !RFC1918(etc).
Use the tool for the job and don't pontificate (unless you know best!)
https://en.wikipedia.org/wiki/Berkeley_r-commands
They do the job, quite well.
I suggest that you use them for all your production needs, exclusively!
You get rid of some (wannabe-)greybeard nerd sniping someone's technical blog about an obscure linux util? What would even be left here? Just B2B SaaS AI blah blah forever?
You wan't them to couch this in "great job! Use whatever your comfortable with!" Are we really at that point?
What makes it a better drop in replacement?
-PuTTY pscp allows raw passwords on the command line, or from a file. OpenSSH is unreasonable in refusing to do this.
-Scripting can adapt to a .netrc easily; OpenSSH will never do this.
-Modern OpenSSH is a nightmare when using legacy crypto, while pscp is fluid. There is nothing wrong with hmac-md5, and no reason to refuse it. I will take PuTTY or dropbear in a heartbeat over these burned bridges and workarounds.
https://www.openssh.org/legacy.html
-pscp does not link to dozens of libraries as ssh/scp does, so it is easier to build with less dependency. The ldd output of ssh and scp on rhel9 is 23 libraries, while PuTTY is 3 [package obtained from EPEL].
-pscp strongly leans to SFTP on the backend and can be directed to use it exclusively, so there is no ambiguity.
-Using pscp with a retry on fail is much easier than sftp -b.
-The wacky cipher control on rhel8 does not impact the PuTTY tools.
That is an extensive list.
You can use `sshpass` to force it through a command line argument. However, arguments can be viewed by any process through `/proc`, `ps`, etc. It's pretty reasonable to not support exposure of the password like that, especially since you can force it through using another tool if you really, really need to.
It is not reasonable to insist on keys for batch use.
Not at all.
From the outset SecSH (SSHv2, the thing you actually use today and if you're younger, likely the only thing you ever have used) has public key authentication as a Mandatory To Implement feature. Implementations where that doesn't work aren't even SSH, they're garbage.
I do not have a choice!
This grew out of FTP less than a decade ago. Everyone has always known password auth; it cannot die.
Are you on the same planet as the rest of us?
Wouldn't tar do the exact same thing to that file's permissions?
You get the benefit of being able to e.g. get your last download off your desktop to your laptop like this:
or this if you're on bash: or pull a file from a very nested project dir for which you have setup dynamic directories (or shell variables if you're on bash): Just don't pull files from an SCP server that may be malicious. Use on trusted servers. If you do the following on your home dir: That may overwrite .ssh/authorized_keys, .zshrc, etc. because `foo/` is server-side shell code. The client can't say that `.zshrc` resulting from the evaluation of `foo/` doesn't make sense, because it might in the remote shell language.> If you need something that SFTP cannot do, then use tar on both sides.
No reason to make things inconvenient between personal, trusted computers, just because there may be malicious servers out there where one has no reason to SCP.
Something else to note is that your suggestion of using `tar` like `ssh malicious 'tar c foo/' | tar x` faces basically the exact same problem. The server can be malicious and return .ssh/authorized_keys, .zshrc, etc. in the archive for `tar x` to overwrite locally basically exactly the same way. This goes with the point of this SE answer:
> I'd say a lot of Unix commands become unsafe if you consider a MITM on SSH possible. A malicious sudo could steal your password, a malicious communication client could read your mails/instant messages, etc. Saying that replacing scp with sftp when talking to a compromised server will somehow rectify the situation is very optimistic to say the least. [...] In short, if you don't pay attention to which servers you SSH into, there's a high risk for you to be screwed no matter which tools you use, and using sftp instead of scp will be only marginally safer. --- https://unix.stackexchange.com/questions/571293/is-scp-unsaf...
I think this whole problem with SCP just stems from not having properly documented this aspect in the manpage, so people expected it to just take filepaths.
signed -confused
I had it wrapped in stunnel TLS, but I ripped that out recently as I am retiring and the new staff is simply not capable of maintaining that configuration.
My users were yelling, and the patch to tinysshd to omit all permissions checks silenced the complaints. No, it's not pretty.
but also... who has a dir with 777 permissions? Is that something people do nowadays?
Script all the things. double-check your scripts... always be backing up.
I would have used AutoHotkey or something similar in such a scenario.
If you still have some access (console, password login, another sudo user), this usually fixes it:
(optional, if the user needs sudo) Not to shill too hard, but this exact "keys/perms/sudo drift" failure mode is why Userify exists (est. 2011): local accounts on every box + a tiny outbound-only agent that polls and overwrites desired state (keys, perms, sudo role). If scp/rsync/deploy steps clobber stuff, the next poll re-converges it (cloud default ~90s; self-host default ~10s; configurable). Removing a user also kills their sessions. No inbound ports to nodes, no PAM/NSS hooks, auditable.Shim (old but readable): https://github.com/userify/shim/blob/master/shim.py#L308 (obligatory): https://userify.com
Too many burned fingers to not do this little dance almost every other time.
Actually, I lied, I just use rsync like an insane person.
<bleep> that nonsense!
Perhaps you got bot flagged or something