whatacold/ppcompile: An Emacs package to ease the development workflow of coding-locally-and-compiling-remotely

This Emacs package tries to ease the development workflow that consists of coding locally, compiling remotely, and fixing errors with next-error locally, which is the typical development cycle for C/C++ projects.

It rsync’s (“ping”) current project to a remote machine, compiles it there, and convert remote paths in the *compilation* output to local paths (“pong”) according to a configured mapping alist so that compilation-mode can find the files containing errors or warnings correctly while executing M-x next-error and M-x previous-error.

+---------+                                  +--------+
|         |              ping: rsync         |        |
|         | -------------------------------> |        |
|  local  |                                  | remote |
|         | <------------------------------- |        |
|         |   pong: compile & convert paths  |        |
+---------+                                  +--------+

The project root is detected via (project-current), if it fails ppcompile will take the top-level git directory as project root.

Currently, it’s only tested with GNU Emacs 26.3 on Linux.

Dependencies

• built-in packages: compile, auth-source, project, and files-x
• external programs: rsync, ssh, expect.

  Make sure you have these on your system. It will avoid using expect if expect isn’t available on your system, so SSH public key authentication is mandatory in this case.

How to Use

• M-x ppcompile-config-project to configure various settings for a project.

  This command’s detail can be found in the section Configurations.

• M-x ppcompile-ping to rsync the project.
• M-x ppcompile-pong to compile the project remotely.

  It respects the compilation-read-command variable of compile, which means it will give you a chance to edit the compile command if you set it to non-=nil=. The input history is kept in ppcompile--remote-command-history, you can hit M-n and M-p to navigate the history list.

  If you want to change the prompt behavior temporarily, prefix the command, i.e. C-u M-x ppcompile-pong.

• M-x ppcompile to ping-pong compile the current project.

  It rsync’s the current project to the remote host, compiles it remotely, takes back the output and converts the remote paths to the local ones, so that compilation-mode works perfectly.

  This command simply executes ppcompile-ping and ppcompile-pong sequentially. The command prompt behavior is exactly same as the M-x ppcompile-pong command.

• M-x ppcompile-toggle-debug to toggle debugging.
• M-x ppcompile-get-ssh-password to get the password of the current project, if password authentication is used, this command is intended for debugging.

Configurations

There is quite some configuration to set, globally or per project.

M-x ppcompile-config-project will guide you through to set them up in .dir-locals.el in the project root. It makes life a little bit easier if you have more than one remote hosts to compile different projects, which often is the case. You can then fine-tune the .dir-locals.el file after finishing the command.

The input history is kept in the variable ppcompile--config-history, so you can hit M-n and M-p to get your previous input to save you some effort.

The configurations covered by that command include:

ppcompile-ssh-hostthe remote host ppcompile-ssh-portthe ssh port of remote host, which defaults to 22 ppcompile-ssh-useruser name, which defaults to currently logged in user, as returned by (user-login-name) ppcompile-rsync-dst-dirremote containing directory for the project ppcompile-remote-compile-commandThe compile command executed under the remote project directory. ppcompile-path-mapping-listalist for path mapping

The car of each whose element is a remote path, and the cdr a local path, all paths should be absolute paths.

The above configuration often may vary from projects to projects, they are often set per project. There are also other global configurations, which have defaults:

ppcompile-ssh-additional-argsadditional arguments for the ssh command line ppcompile-rsync-additional-argsadditional arguments for the rsync command line ppcompile-rsync-exclude-lista list specifying files you want to exclude, such as binary files. ppcompile-ssh-executableThe ssh executable ppcompile-rsync-executableThe rsync executable ppcompile-expect-executableThe expect executable ppcompile-with-password-script-pathThe path of the helper expect script with-password.exp.

The default value may be wrong if your .elc file isn’t in the same directory of the with-password.exp, which means the file path doesn’t exist, to make SSH public key authentication mandatory.

Besides that, you may need to configure your passwords in some auth-source backends, for example, one entry per host in ~/.authinfo looks like:

machine localhost port 22 login try password 1

Also, pay attention to Emacs variable auth-sources to include your setting.

That being said, public key authentication is recommended thought, whenever it’s possible, and keep various configurations including identity files in ~/.ssh/config. (Manage SSH connections with =~/.ssh/config=)

Troubleshooting

After the above settings, chances are that it still doesn’t work. You can troubleshoot it by following these steps:

1. Turn on the debugging flag by M-x ppcompile-toggle-debug

   Re-run it once again, and check out the shell commands in the *Message* buffer, and if there is setting wrong. Run the command on a terminal manually, to see if there is more error info.

2. Confirm that the password is right by M-x ppcompile-get-ssh-password if you’re using password authentication for ssh.

   Setting auth-source can be tricky, so this may help.

Note that these commands should be executed on the buffers of project files, to take advantage of the configurations for that particular project.

Other Solutions

• sshfs mounts the remote FS locally, which would be an option if you have a stable, fast network and want to edit remote files just like locally.

  Note that you still need to compile it on the remote host, though you can edit it within your local environment.

• mainframer, a tool for remote builds, although not based on Emacs, is a more general solution with a similar idea.

• Run make test to test the code

  And make test-with-sshd to test the functionality with a sshd server, which requires some additional setup:

  1. Start a ssh server at port 22000: /usr/sbin/sshd -p 22000
  2. Copy the public key file: ssh-copy-id -p 22000 -i ./test/id_ppcompile_test localhost This will append the public key file to ~/.ssh/authorized_keys, so don’t do this on your publicly available server, because it will be open to anyone who uses the private key in the test/ directory to ssh into your server, and do something evil.
• make checkdoc checks the docstrings.
• make compile compiles the elisp files.

Final words

This was my first time to roll out a package seriously, I believe there is much to improve, so pull requests and issues are very welcome.