Self-hosted ML deployment platform
source link: https://www.tuicool.com/articles/jMvI7n7
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Get started: Install • Tutorial • Docs • Examples
Learn more: Website • Blog • Subscribe • Contact
Cortex is a machine learning model deployment platform that runs in your AWS account. You define deployments with simple declarative configuration and Cortex deploys your models as JSON APIs. It also handles autoscaling, rolling updates, log streaming, inference on CPUs or GPUs, and more.
Cortex is actively maintained by a venture-backed team of infrastructure engineers and we're hiring .
How it works
Defineyour deployment using declarative configuration:
# cortex.yaml - kind: api name: my-api model: s3://my-bucket/my-model.zip request_handler: handler.py compute: min_replicas: 5 max_replicas: 20
Customizerequest handling (optional):
# handler.py def pre_inference(sample, metadata): # Python code def post_inference(prediction, metadata): # Python code
Deployto your cloud infrastructure:
$ cortex deploy Deploying ... https://amazonaws.com/my-api # Your API is ready!
Servereal time predictions via scalable JSON APIs:
$ curl -d '{"a": 1, "b": 2, "c": 3}' https://amazonaws.com/my-api { prediction: "def" }
Spinning up a Cortex cluster on your AWS account
# Download the install script $ curl -O https://raw.githubusercontent.com/cortexlabs/cortex/master/cortex.sh && chmod +x cortex.sh # Set your AWS credentials $ export AWS_ACCESS_KEY_ID=*** $ export AWS_SECRET_ACCESS_KEY=*** # Provision infrastructure on AWS and install Cortex $ ./cortex.sh install # Install the Cortex CLI on your machine $ ./cortex.sh install cli
Key features
-
Minimal declarative configuration:Deployments can be defined in a single
cortex.yaml
file. -
Autoscaling:Cortex can automatically scale APIs to handle production workloads.
-
Multi framework:Cortex supports TensorFlow, Keras, PyTorch, Scikit-learn, XGBoost, and more.
-
Rolling updates:Cortex updates deployed APIs without any downtime.
-
Log streaming:Cortex streams logs from your deployed models to your CLI.
-
CPU / GPU support:Cortex can run inference on CPU or GPU infrastructure.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK