MLOps Platform

Main Concepts of EverlyAI

Working with EverlyAI is simple. To run any code, model training, serving etc, you create a project. When creating a project, we need to select region and GPU type, the job type and the code type.

Job type

Job type tells EverlyAI how to run your code. The differences of different job types are shown below.

Model TrainingModel Serving
Optimized forBatch JobsOnline Jobs
When your job crashes or endsEverlyAI automatically stops your projectEverlyAI will restart the job

TL;DR Use Model Serving for any web application and Model Training for offline jobs, such as model training and data processing.

Code type

EverlyAI supports both prebuilt docker images and any local code, that is, any scripts or python code. When we create a project with local code, we need to add a file at the root directory. When EverlyAI transfers control to your code, it will call the file.


When we create a project on EverlyAI, it will bring up an instance to run our job. We can think of an instance as equivalent to a physical GPU machine. The instance will go through a few states before it runs our code.

Instance state


Our multi-cloud scheduler has decided where to place your instances.

Instance Started

We have brought up the machine on the cloud.

Server Started

We have launched the server on the GPU instance. We will transfer the control to your code. If the job type is model_training, the state will become Running now. If the job type is model_serving, we will probe the readiness of your web server. When the endpoint is ready, the state will become Running.

Server Running

All initialization has completed and your code is in full control now.