Emacs TRAMP

Table of Contents

Author Lee Hinman (lee@writequit.org)
Date 2017-06-20 19:36:02

1 Introduction

TRAMP is a package providing an abstraction layer that can be used for accessing remote files on different machines. I say "abstraction layer" because it's not just a simple library for reading and writing files, it hooks into Emacs at a low enough level that other packages need not be aware of it in order to use it.

TRAMP stands for Transparent Remote (file) Access, Multiple Protocol

So what is this and how do I use it?

2 What's required to use TRAMP

While TRAMP is built in to Emacs, it does require some external tools, so if you can, install these:

  • ssh
  • scp
  • nc (optional)
  • rsh/rlogin/telnet (if you hate security)
  • ftp (optional)
  • mimencode or uuencode (both optional)
  • smbclient (optional)

And if you're on Windows:

  • plink
  • psch

Which are part of the Putty suite.

3 How to use TRAMP

Tramp has a special syntax that you use when opening files in order to access the file as if it were local:

/{method}:{user}@{host}#{port}:{remote-file-path}

Not all the parts are required, so we'll look at a lot of examples

4 TRAMP methods

There are a bunch of methods that you can use with TRAMP, here are some of them

Method Name Description
ssh The most common method, uses ssh
scp Uses both ssh and scp to transfer data
rsync Uses ssh to connect and rsync to transfer data
su Use su to edit a file as a different user
sudo Edit the file using sudo to appear as root
plink Uses plink.exe -ssh to log in to the host
pscp Uses plink to connect and pscp to transfer files
smb Connect to a Samba (Windows file sharing) share, requires smbclient
rsh Don't use this unless you like getting hacked
telnet Don't use this unless you like getting hacked
ftp Don't use this unless you like getting hacked
adb Connect to an Android phone using adb

Most of the time, if you are on OS X or Linux, you'll want to use ssh or scp, and plink if you're on windows. If you really want to see the whole list, you can check tramp-methods.

4.1 Configuration of methods

You can change the method that tramp uses by default by customizing tramp-default-method, for instance, I've set my method default to ssh:

tramp-default-method
"ssh"

You can also set per-host defaults with tramp-default-method-alist, in the format HOST USER METHOD

tramp-default-method-alist
((nil "%" "smb")
 ("\\`\\(127\\.0\\.0\\.1\\|::1\\|localhost6?\\|thulcandra\\)\\'" "\\`root\\'" "su")
 (nil "\\`\\(anonymous\\|ftp\\)\\'" "ftp")
 ("\\`ftp\\." nil "ftp"))

If you also want to change the default username that tramp uses to connect, as by default it uses your current username, you can change tramp-default-user and tramp-default-user-alist for that.

Finally, if you go to a remote host quite often, you can make it the default by setting tramp-default-host or tramp-default-host-alist.

5 Editing remote files

Imagine you have a server remoteFoo and you want to edit a file in /etc/hosts, first you would invoke find-files with C-x C-f, and then type:

/ssh:user@remoteFoo:/etc/hosts

Hit enter, and now you have a buffer that coordinates to this file. You can also leave out the "user" part if you use the same user as your current username:

/ssh:remoteFoo:/etc/hosts

Whenever you save the file, tramp will transfer it back to the remote machine, you're technically editing a buffer that is a local (temporary) file, which is being transferred back and forth as needed by Emacs.

6 Troubleshooting

You can turn on tramp's logging by setting tramp-verbose to a number between 0 and 10 (inclusive), the default value is 3, with increasing values being more and more verbose:

Value Description
0 silent (no tramp messages at all)
1 errors
2 warnings
3 connection to remote hosts (default level)
4 activities
5 internal
6 sent and received strings
7 file caching
8 connection properties
9 test commands
10 traces (huge).

Sometimes you run into trouble with tramp, it hangs, or it can't connect to a machine, this can especially happen if you forcibly disconnect the connection (like closing your laptop lid) while a session is open, in that case, you might need to clean up tramp's internal accounting before opening a new tramp session.

There are two "magic" cleanup commands:

  • tramp-cleanup-all-buffers

    This kills all the remote buffers that tramp has around

  • tramp-cleanup-all-connections

    Flushes all of tramp's internal things, like password, file, and connection caches, as well as internal buffers.

6.1 Dealing with funky shells

Tramp does a pretty decent job of handling shells, but there are a couple of exceptions. The first one is Fish, which is a non-posix shell, so it doesn't handle things as nicely. I haven't tested it very much, so your mileage may vary.

The other issue is with prompts, tramp tries to figure out whether the connection is working by issuing a couple of commands like ls and cd, but weird prompts can sometimes confuse it. If you have a seriously crazy prompt, check out shell-prompt-pattern and tramp-shell-prompt-pattern if you have any problems.

7 Editing local files as a different user

Have you ever accidentally opened a file as your user when in order to change it, you have to be root? Tramp can help!

(defun edit-file-with-sudo ()
  "Take the file currently being edited, and open it as root with `sudo'."
  (interactive)
  (let ((file-name (buffer-file-name)))
    (when file-name
      (find-alternate-file (concat "/sudo::" file-name)))))

This will re-open the buffer you're currently editing, but as root.

If you didn't want to be "root", but a different user, you could use the "su" method instead, with a username, so C-x C-f and then:

/su:test@localhost:/home/test/hi

The "sudo" method might allow you to do this also, but "su" saves two whole characters!

8 SSH config magic

Now, it might not sound very efficient to make a connection, and in fact, it's not, this is why tramp caches a bunch of things by using ssh options to keep a connection around. It does this when tramp-use-ssh-controlmaster-options is set to "t", which it is by default.

Tramp will use things like ControlMaster, ControlPath, and ControlPersist to keep a connection around for a certain time.

In some cases though, you don't want this, you want to use your configuration options. Things that you stuck in ~/.ssh/config instead of tramps. If you set tramp-use-ssh-controlmaster-options to "nil" it will use those options, here are the ones I recommend:

Host *
  ControlMaster auto
  ControlPath ~/.ssh/master-%r@%h:%p
  # ControlPersist 30
  ForwardAgent yes
  ServerAliveInterval 60

ControlMaster allows sharing multiple sessions over a single network connection, "auto" means to try and use a master connection, or create one if it doesn't exist. This is what can speed up tramp.

ControlPath is the path for the socket user for the master control.

ControlPersist keeps the connection open for a certain amount of time, "0" or "yes" keeps it open forever, while a number means the number of seconds before automatically closing it.

Usually then, I'll "persist" a connection by ssh-ing to the machine in a terminal or shell buffer, and then tramp doesn't have to re-negotiate the connection every time I make a change to a buffer, it just has to transfer the file.

9 Integrating with other parts of Emacs

Since tramp isn't just for editing files, there are a lot of really neat things you can do with it. This isn't an exhaustive list, these are just the ones I use the most often.

9.1 Dired

We talked about using Dired as a file manager a while back, and that works even better with remote dired buffers.

Pop open a dired buffer with C-x d (or M-x dired) and you can browse a remote directory just like you can edit files,

/ssh:ivalice:/home/hinmanm/

Even better, if you open two dired buffers next to each other and set dired-dwim-target to "t", you can use Emacs as an ssh-client, to transfer files between your local machine and a remote machine.

tramp-dired1.png

Figure 1: selecting files, remote dired on the right

tramp-dired2.png

Figure 2: hitting "C" to copy the marked files from my local machine to the remote server

tramp-dired3.png

Figure 3: the files have been copied to the server

9.2 Eshell

You're likely familiar with ssh-ing into a node and executing commands, but how does this work for eshell?

Well, turns out you can execute arbitrary commands using tramp also, in your eshell buffer, do the following ("writequit.org" is my remote machine). Here's an example:

~ λ cd /ssh:writequit.org:/home/hinmanm/www/

@writequit.org /s/home/hinmanm/www λ ls
total 22M
drwxr-xr-x   3 hinmanm        hinmanm  4.0k 2016-03-31  2016 articles
-rw-r--r--   1 hinmanm        hinmanm  3.7M 2015-06-02  2015 articles.tgz
-rw-rw-r--   1 hinmanm        hinmanm  5.1k 2016-08-19  2016 bibliography.html
-rw-rw-r--   1 hinmanm        hinmanm  1.9k 2017-04-02 20:55 bibliography.org
drwxr-xr-x  18 hinmanm        hinmanm  4.0k 2015-05-12  2015 blog
-rw-r--r--   1 hinmanm        hinmanm   18M 2015-06-02  2015 blog.tgz
drwxrwxr-x   3 hinmanm        hinmanm  4.0k 2016-09-28  2016 css
-rw-rw-r--   1 hinmanm        hinmanm  4.9k 2016-09-07  2016 decentralize.org
drwxrwxr-x   3 hinmanm        hinmanm  4.0k 2016-06-21  2016 denver-emacs
drwxrwxr-x   6 hinmanm        hinmanm  4.0k 2017-03-12 11:41 elastic-downloads
drwxrwxr-x   3 hinmanm        hinmanm  4.0k 2017-04-05 21:42 eos
-rw-r--r--   1 hinmanm        hinmanm  6.8k 2015-06-02  2015 favicon.ico
drwxrwxr-x   2 hinmanm        hinmanm  4.0k 2016-11-10  2016 files
-rw-rw-r--   1 hinmanm        hinmanm    20 2016-05-25  2016 foo.php
-rw-r--r--   1 hinmanm        hinmanm   940 2015-06-02  2015 github.css
drwxrwxr-x   2 hinmanm        hinmanm  4.0k 2016-04-26  2016 images
-rw-rw-r--   1 hinmanm        hinmanm   20k 2017-02-15 17:05 index.html
-rw-rw-r--   1 hinmanm        hinmanm   11k 2016-09-02  2016 index.org
-rw-rw-r--   1 hinmanm        hinmanm   15k 2017-02-28 13:45 jenkins-jdk.png
-rw-rw-r--   1 hinmanm        hinmanm  2.9k 2016-08-17  2016 keybase.txt
-rw-r--r--   1 hinmanm        hinmanm  4.0k 2015-06-02  2015 master.css
drwxr-xr-x   4 hinmanm        hinmanm  4.0k 2015-06-02  2015 misc
-rw-r--r--   1 hinmanm        hinmanm  489k 2015-06-02  2015 misc.tgz
drwxrwxr-x   5 hinmanm        hinmanm  4.0k 2017-05-25 10:29 org
drwxr-xr-x   3 hinmanm        hinmanm  4.0k 2015-06-02  2015 papers
drwxrwxr-x   3 hinmanm        hinmanm  4.0k 2016-02-08  2016 paste
-rw-rw-r--   1 hinmanm        hinmanm  6.7k 2015-12-12  2015 posts.html
-rw-rw-r--   1 hinmanm        hinmanm  5.4k 2015-12-12  2015 posts.xml
drwxr-xr-x  10 hinmanm        hinmanm  4.0k 2015-06-02  2015 projects
drwxrwxr-x   2 technomancy    technoma 4.0k 2016-09-19  2016 technomancy

@writequit.org /s/home/hinmanm/www λ echo "<h1>Hi There!</h1>" > hello-from-tramp.html

@writequit.org /s/home/hinmanm/www λ cd

~ λ curl -L writequit.org/hello-from-tramp.html
<h1>Hi There!</h1>

~ λ

The nice thing about this is that all of your typing is local, if it's a slow connection (or if there is no connection because you've been disconnected), tramp only sends things over the network when you hit RETURN, not when you press any key (like ssh does).

9.3 Magit

You can do everything you usually would with Magit, but over tramp on a remote machine! Either invoke magit-status on a remote buffer (one you already opened with tramp), or manually with M-: (magit-status "/ssh:ivalice:/home/hinmanm/es/elasticsearch") (or whatever location.)

9.4 Org-mode

Want do execute some org source blocks, but on a remote machine? You can!

Simply do:

#+BEGIN_SRC sh :dir /ssh:root@ivalice:/etc
cat passwd
#+END_SRC

Or with a session (doesn't show up in HTML output):

ssh ivalice
hostnamectl
 Static hostname: ivalice
       Icon name: computer-desktop
         Chassis: desktop
      Machine ID: e2e5c6fd949748a3b98d545b50d69de9
         Boot ID: 8d3c7ed1f1f14627b9facbf842eff6ed
Operating System: Fedora 25 (Workstation Edition)
     CPE OS Name: cpe:/o:fedoraproject:fedora:25
          Kernel: Linux 4.11.3-200.fc25.x86_64
    Architecture: x86-64

There's a great description of this in Howard's Literate Devops, with many more examples.

10 Connecting to Android phones

As a bonus, here's a non-network connection, where I can look at the contents of my phone using tramp:

sudo adb devices
List of devices attached
LGD851d5b7eb08	device

And then I should be able to connect with this:

/adb::/

(DEMO)

11 Lee's tramp configuration

In case it's useful to anyone, here's my configuration, it's pretty minimal.

(use-package tramp
  :defer 5
  :config
  (with-eval-after-load 'tramp-cache
    (setq tramp-persistency-file-name "~/.emacs.d/tramp"))
  (setq tramp-default-method "ssh"
        tramp-default-user-alist '(("\\`su\\(do\\)?\\'" nil "root"))
        tramp-adb-program "adb"
        ;; use the settings in ~/.ssh/config instead of Tramp's
        tramp-use-ssh-controlmaster-options nil
        ;; don't generate backups for remote files opened as root (security hazzard)
        backup-enable-predicate
        (lambda (name)
          (and (normal-backup-enable-predicate name)
               (not (let ((method (file-remote-p name 'method)))
                      (when (stringp method)
                        (member method '("su" "sudo"))))))))

  (use-package tramp-sh
    :config
    (add-to-list 'tramp-remote-path "/usr/local/sbin")
    (add-to-list 'tramp-remote-path "/opt/java/current/bin")
    (add-to-list 'tramp-remote-path "/opt/gradle/current/bin")
    (add-to-list 'tramp-remote-path "~/bin")))

Author: Lee Hinman

Created: 2017-06-20 Tue 19:36