Model Server for Apache MXNet (MMS)
Multi Model Server (MMS) is a flexible and easy to use tool for serving deep learning models trained using any ML/DL framework.
Use the MMS Server CLI, or the pre-configured Docker images, to start a service that sets up HTTP endpoints to handle model inference requests.
Features
Serving Quick Start – Basic server usage tutorial
Model Archive Quick Start – Tutorial that shows you how to package a model archive file.
Installation – Installation procedures and troubleshooting
Serving Models – Explains how to use multi-model-server.
REST API – Specification on the API endpoint for MMS
Model Zoo – A collection of MMS model archive (.mar) files that you can use with MMS.
Packaging Model Archive – Explains how to package model archive file, use model-archiver.
Docker – How to use MMS with Docker and cloud services
Logging – How to configure logging
Metrics – How to configure metrics