48

Pichi – HTTP/Socks5/SS Proxy, Whose Route Controlled via APIs

 5 years ago
source link: https://www.tuicool.com/articles/hit/Y7nUbeY
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

Pichi

Pichi is an application layer proxy, which can be fully controlled via RESTful APIs.

Build Status

OS macOS 10.13 Alpine 3.8 Windows 10 iOS 12.1 Android 9 Toolchain Xcode 10.1 GCC 8.x VC++2017 Xcode 10.1 NDK r18b Architecture x86_64 x86_64 x86_64 arm64/arm64e arm64 Status

Using Pichi API

Resources

  • Ingress : defines an incoming network adapter, containing protocol, listening address/port and protocol specific configurations.
  • Egress : defines an outgoing network adapter, containing protocol, next hop address/port and protocol specific configurations.
  • Rule : contains a group of conditions, such as IP ranges, domain regular expressions, the countries of the destination IP, and so on, that the incoming connection matching ANY conditions means the one matching this rule.
  • Route : indicates a priority ordered sequence of [rule, egress] pairs, and a default egress which would be forwarded to if none of the rules matched.

API Specification

Pichi API

Examples

Proxy like ss-local(shadowsocks-libev)

$ curl -i -X PUT -d '{"type":"socks5","bind":"127.0.0.1","port":1080}' http://pichi-router:port/ingresses/socks5
HTTP/1.1 204 No Content

$ curl -i -X PUT -d '{"type":"ss","host":"my-ss-server","port":8388,"method":"rc4-md5","password":"my-password"}' http://pichi-router:port/egresses/shadowsocks
HTTP/1.1 204 No Content

$ curl -i -X PUT -d '{"default":"shadowsocks"}' http://pichi-router:port/route
HTTP/1.1 204 No Content

HTTP proxy except intranet

$ curl -i -X PUT -d '{"type":"http","bind":"::","port":8080}' http://pichi-router:port/ingresses/http
HTTP/1.1 204 No Content

$ curl -i -X PUT -d '{"type":"http","host":"http-proxy","port":8080}' http://pichi-router:port/egresses/http
HTTP/1.1 204 No Content

$ curl -i -X PUT -d '{"range":["::1/128","127.0.0.1/32", "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "fc00::/7"],"domain":["local"],"pattern":["^localhost$"]}' http://pichi-router:port/rules/intranet
HTTP/1.1 204 No Content

$ curl -i -X PUT -d '{"default":"http","rules":[["intranet","direct"]]}' http://pichi-router:port/route
HTTP/1.1 204 No Content

100 shadowsocks servers

$ for((i=20000;i<20100;++i)); do \
>   curl -X PUT \
>   -d "{\"type\":\"ss\",\"bind\":\"::\",\"port\":$i,\"method\":\"rc4-md5\",\"password\":\"pw-$i\"}" \
>   "http://pichi-router:port/ingresses/$i"; \
> done

dark web

$ curl -i -X PUT -d '{"type":"socks5","host":"localhost","port":9050}' http://pichi-router:port/egresses/tor
HTTP/1.1 204 No Content

$ curl -i -X PUT -d '{"type":"http","host":"localhost","port":4444}' http://pichi-router:port/egresses/i2p
HTTP/1.1 204 No Content

$ curl -i -X PUT -d '{"domain":["onion"]}' http://pichi-router:port/rules/onion
HTTP/1.1 204 No Content

$ curl -i -X PUT -d '{"domain":["i2p"]}' http://pichi-router:port/rules/i2p
HTTP/1.1 204 No Content

$ curl -i -X PUT -d '{"rules":[["onion","tor"],["i2p","i2p"]]}' http://pichi-router:port/route
HTTP/1.1 204 No Content

socks5 server with TLS certificate issued by Let's encrypt CA

$ curl -i -X PUT -d '{"type":"socks5","bind":"::1","port":1080, \
      "tls":true, \
      "key_file": "/etc/letsencrypt/live/example.com/privkey.pem", \
      "cert_file": "/etc/letsencrypt/live/example.com/fullchain.pem" \
    }' http://pichi-router:port/ingresses/socks5s
HTTP/1.1 204 No Content

Supported protocols

Ingress protocols

Egress protocols

  • HTTP Proxy: defined by RFC 2068
  • HTTP Tunnel: defined by RFC 2616
  • SOCKS5: defined by RFC 1928
  • Shadowsocks: defined by shadowsocks.org
  • Direct: connecting to destination directly
  • Reject: rejecting request immediately or after a fixed/random delay

NOTE:HTTP egress would like to try HTTP CONNECT first. HTTP proxy will be chosen if the previous handshake is failed.

Build

Requirements

CMake options

  • BUILD_SERVER : Build pichi application, the default is ON .
  • BUILD_TEST : Build unit test cases, the default is ON .
  • STATIC_LINK : Generate static library, the default is ON .
  • INSTALL_HEADERS : Install header files, the default is OFF .
  • ENABLE_TLS : Provide TLS support, the default is ON .

Build and run tests

Build and run on Unix-like:

$ cmake -B build .
$ cmake --build build
$ cmake --build build --target test

Build and run on Windows with Vcpkg : 

PS C:\pichi> cmake -DCMAKE_TOOLCHAIN_FILE=C:/vcpkg/scripts/buildsystems/vcpkg.cmake `
>> -DVCPKG_TARGET_TRIPLET="x64-windows-static" -DCMAKE_BUILD_TYPE=MinSizeRel -B build .
PS C:\pichi> cmake --build build --config MinSizeRel
PS C:\pichi> cmake --build build --config MinSizeRel --target test

Docker

The pre-built docker image can be found on Docker Hub , which is automatically generated according to docker/pichi.dockerfile . Furthermore, docker/builder.dockerfile is intended to provide a docker environment for development. 

$ docker pull pichi/pichi
$ docker run --rm pichi/pichi pichi -h
Allow options:
  -h [ --help ]              produce help message
  -l [ --listen ] arg (=::1) API server address
  -p [ --port ] arg          API server port
  -g [ --geo ] arg           GEO file
  --json arg                 Initail configration(JSON format)
  -d [ --daemon ]            daemonize
  -u [ --user ] arg          run as user
  --group arg                run as group
$ docker run -d --name pichi --net host --restart always pichi/pichi \
>   pichi -g /usr/share/pichi/geo.mmdb -p 1024 -l 127.0.0.1
c51b832bd29dd0333b0d32b0b0563ddc72821f7301c36c7635ae47d00a3bb902
$ docker ps -n 1
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
c51b832bd29d        pichi/pichi         "pichi -g /usr/share…"   1 seconds ago       Up 1 seconds                            pichi

Build library for iOS/Android

Pichi is designed to run or be embeded into some APPs on iOS/Adnroid. deps-build directory gives some helping scripts to build Pichi's dependencies for iOS/Android.

iOS

It's very simple to build a C/C++ project managed by CMake, if CMAKE_TOOLCHAIN_FILE is set to ios.toolchain.cmake . 

$ cmake -D CMAKE_TOOLCHAIN_FILE=/path/to/ios.toolchain.cmake \
>   -D IOS_PLATFORM=OS -D IOS_ARCH=arm64 [other options] /path/to/project

On the other hand, deps-build/boost.sh can generate libraries for iOS if below environment variables are set:

  • PLATFORM : to specify target OS( iphoneos, iphonesimulator, appletvos, appletvsimulator );
  • IOS_ROOT : to specify root install directory of headers/libraries;
  • ADDRESS_MODEL : to specify address bits of target CPU(64 or 32).

For example:

$ # In macOS with Xcode 10.0 or above
$ export PLATFORM=iphoneos
$ export IOS_ROOT=/path/to/ios/root
$ export ADDRESS_MODEL=64
$ bash deps-build/boost.sh
Usage: boost.sh <src path>
$ bash deps-build/boost.sh /path/to/boost
...

Android

The usage of deps-build/boost.sh is very similar to iOS one, except environment varialbes:

  • PLATFORM : android-<API>s are available;
  • ANDROID_ROOT : to specify root install directory of headers/libraries;
  • ADDRESS_MODEL : to specify address bits of target CPU(64 or 32).

Android NDK kindly provides build/tools/make_standalone_toolchain.py script to generate a cross-compiling toolchain for any version of Android. 

$ export NDK_ROOT=/path/to/ndk
$ export TOOLCHAIN_ROOT=/path/to/toolchain
$ export ADDRESS_MODEL=64
$
$ # Create cross toolchain
$ python ${NDK_ROOT}/build/tools/make_standalone_toolchain.py --arch arm64 --api 28 \
>   --stl libc++ --install-dir ${TOOLCHAIN_ROOT}
$
$ # Build boost
$ bash deps-build/boost.sh /path/to/boost
$
$ # Build other dependent libraries
$ cmake -D CMAKE_SYSROOT=${TOOLCHAIN_ROOT}/sysroot \
>   -D CMAKE_INSTALL_PREFIX=${TOOLCHAIN_ROOT}/sysroot \
>   -D CMAKE_C_COMPILER=${TOOLCHAIN_ROOT}/bin/clang \
>   [other options] -B build /path/to/other/libraries
$ cmake --build build --target install
$
$ # Build pichi
$ cmake -D CMAKE_SYSROOT=${TOOLCHAIN_ROOT}/sysroot \
>   -D CMAKE_INSTALL_PREFIX=${TOOLCHAIN_ROOT}/sysroot \
>   -D CMAKE_CXX_COMPILER=${TOOLCHAIN_ROOT}/bin/clang++ \
>   [other options] -B build .
$ cmake --build build --target install

Cross-Compiling for other architecture

deps-build/boost.sh doesn't provide any cross-compiling steps except iOS/Android, but it's not very complicated to cross-compile Boost libraries. There's only one thing important. Boost.Context requires that target-os , abi , binary-format , architecture and address-model must be explicitly specified correctly while cross-compiling .

On the contrary, other libraries can be cross-complied very easily by using CMake. For example:

$ # Cross-Compiling for aarch64-freebsd12.0
$ export AARCH64_SYSROOT=/path/to/aarch64/base/system
$ export CROSS_FLAGS="-target aarch64-unknown-freebsd12.0 --sysroot=${AARCH64_SYSROOT}"
$
$ # Build Boost
$ cd /path/to/boost
$ ./bootstrap.sh
$ cat > project-config.jam <<EOF
> using clang :
> : /usr/bin/clang++ -std=c++17 ${CROSS_FLAGS}
> ;
> EOF
$ ./b2 --with-context --with-filesystem --with-program_options --with-system \
    --prefix=${AARCH64_SYSROOT} target-os=freebsd abi=aapcs binary-format=elf \
    architecture=arm address-model=64 variant=release link=static install
$
$ # Build other libraries
$ cmake -D CMAKE_C_COMPILER=clang -D CMAKE_C_FLAGS="${CROSS_FLAGS}" \
    -D CMAKE_INSTALL_PREFIX=${AARCH64_SYSROOT} -B build /path/to/library
$ cmake --build build --target install
$
$ # Build pichi
$ cmake -D CMAKE_CXX_COMPILER=clang++ -D CMAKE_CXX_FLAGS="${CROSS_FLAGS}" \
    -D CMAKE_INSTALL_PREFIX=${AARCH64_SYSROOT} -B build .
$ cmake --build build
$ file build/server/pichi
build/server/pichi: ELF 64-bit LSB executable, ARM aarch64, version 1 (FreeBSD),
dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 12.0 (1200086),
FreeBSD-style, with debug_info, not stripped

Run pichi server

There are 2 ways to start pichi server:

  • Standalone : pichi server runs in its own process,
  • In-Process : pichi server runs in its supervisor process.

Standalone

Standalone mode requires BUILD_SERVER CMake option, which builds code in server directory. For example: 

$ cmake -D CMAKE_INSTALL_PREFIX=/usr -D CMAKE_BUILD_TYPE=MinSizeRel -D BUILD_SERVER=ON -B build .
$ cmake --build build --target install/strip
$ /usr/bin/pichi -h
Allow options:
  -h [ --help ]              produce help message
  -l [ --listen ] arg (=::1) API server address
  -p [ --port ] arg          API server port
  -g [ --geo ] arg           GEO file
  --json arg                 Initail configration(JSON format)
  -d [ --daemon ]            daemonize
  -u [ --user ] arg          run as user
  --group arg                run as group

--port and --geo are mandatory. --json option can take a JSON file as an Initial configuration to specify ingresses/egresses/rules/route. The initial configuration format looks like: 

{
  "ingresses": {
    "ingress-0": {/* ingress configuration */},
    "ingress-1": {/* ingress configuration */}
  },
  "egresses": {
    "egress-0": {/* egress configuration */},
    "egress-1": {/* egress configuration */}
  },
  "rules": {
    "rule-0": {/* rule configuration */},
    "rule-1": {/* rule configuration */}
  },
  "route": {/* route configuration */}
}

Furthermore, Pichi server also reload JSON configuration on SIGHUP received if OS supports.

In-Process

In-Process mode is suitable for the scenarios that the standalone process is prohibited or unnecessary, such as iOS/Android, or the supervisor prefers to run pichi in its own process. There are 2 types of interface to run pichi.

C function

C function can be invoked by lots of program languages. It's defined in include/pichi.h : 

/*
 * Start PICHI server according to
 *   - bind: server listening address, NOT NULL,
 *   - port: server listening port,
 *   - mmdb: IP GEO database, MMDB format, NOT NULL.
 * The function doesn't return if no error occurs, otherwise -1.
 */
extern int pichi_run_server(char const* bind, uint16_t port, char const* mmdb);

pichi_run_server will block the calling thread if no error occurs.

C++ class

C++ class might sometimes be friendly while the supervisor is written in C++. It's defined in include/pichi/api/server.hpp : 

class Server {
public:
  Server(boost::asio::io_context&, char const* mmdb);
  void listen(std::string_view bind, uint16_t port);
};

pichi::api::Server accepts a boost::asio::io_context object reference, which is shared by the supervisor. Furthermore, Server::listen doesn't block the calling thread. It means that the supervisor can invoke io_context::run() right where it wants to do. Here's a simple code snippet: 

#include <pichi/api/server.hpp>

auto io = boost::asio::io_context{};

auto server = pichi::api::Server{io, mmdb};
server.listen(bind, port);

// Setup other ASIO services

io.run();  // Thread blocked

Donation

r2qUj2F.png!web


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK