MLOps Platform
Run Any Code

Run Any Code

Everly AI allows you to run any code. You can run any pre-built docker image and non-containerized local code without any modification.

Pre-built docker image

To run any pre-built docker image, simply convert the command to EverlyAI configuration. Let's use the Huggingface TGI as an example. To run it, we can use the following command. The translation is shown in the figure below.

docker run \
  --gpus all \
  --shm-size 1g \
  -p 8080:80 \
  ghcr.io/huggingface/text-generation-inference:1.3 
  --model-id HuggingFaceH4/zephyr-7b-beta
docker

For a complete tutorial on using Huggingface TGI, please refer to this page.

Local code

⚠️

We run your code as USER, not SYSTEM USER or ADMIN. Privileged commands such as sudo or apt-get will fail.

With EverlyAI, you can run any scripts or code as you would locally. In your code directory, add a file everlyai_entrypoint.sh to install dependecies and run the code. An example code directory is shown below.

    • everlyai_entrypoint.sh
    • server.py
    • code.zip

    • Inside the project directory, my_awesome_project, run the following command to generate the code.zip file. And now we are ready to start the project on EverlyAI.

    zip code.zip everlyai_entrypoint.sh server.py

    For a complete tutorial on using local code, please refer to Model demo tutorial.

    File StorageFault Tolerance