Main Concepts of EverlyAI
Working with EverlyAI is simple. To run any code, model training, serving etc, you create a project.
When creating a project, we need to select region and GPU type, the job type and the code type.
Job type
Job type tells EverlyAI how to run your code. The differences of different job types are shown below.
| Model Training | Model Serving | |
|---|---|---|
| Optimized for | Batch Jobs | Online Jobs |
| When your job crashes or ends | EverlyAI automatically stops your project | EverlyAI will restart the job |
TL;DR
Use Model Serving for any web application and Model Training for offline jobs, such as model training and data processing.
Code type
EverlyAI supports both prebuilt docker images and any local code, that is, any scripts or python code. When we create a project
with local code, we need to add a file everlyai_entrypoint.sh at the root directory. When EverlyAI transfers control to your code, it will call
the everlyai_entrypoint.sh file.
Instance
When we create a project on EverlyAI, it will bring up an instance to run our job. We can think of an instance as equivalent to a physical GPU machine. The instance will go through a few states before it runs our code.
Instance state
Scheduled
Our multi-cloud scheduler has decided where to place your instances.
Instance Started
We have brought up the machine on the cloud.
Server Started
We have launched the server on the GPU instance. We will transfer the control to your code.
If the job type is model_training, the state will become Running now.
If the job type is model_serving, we will probe the readiness of your web server. When the endpoint
is ready, the state will become Running.
Server Running
All initialization has completed and your code is in full control now.