File Storage
EverlyAI provides a persistent and fault tolerant file storage. It is optimized for unstructured data. It can be used to
- Store data processing result, which can be used as model training inputs.
- Store model outputs during model training iterations.
- Store servable model weights.
- Store any files which may be useful for your training or serving jobs.
đź’ˇ
Some key features of EverlyAI file storage are
- You do not need to provision file storage or attach the storage to your GPU machine manually. It is available on all machine by default.
- You only pay for the actual storage usage, not provisioned storage capacity.
- The file storage is shared across all machines in the same region. All machines have access to the same file storage.
- You can use developer APIs to manually upload and download files to EverlyAI file storage.
How to use
When an instance starts, the file storage is automatically mounted to the instance at
/everlyai/fs. There is no setup required. Simply write to /everlyai/fs and the file can be
accessed from all instances, developer API (opens in a new tab) and web UI (opens in a new tab).
Also please refer to Stable Diffusion serving tutorial for an example of how to use file storage.
Caveats
- The files are not synchronized across regions. If you create a project in region,
US_WEST, the files are stored in regionUS_WESTas well. It will not be accessible by the instances in another region. If you want to access those files, please copy them across regions. - We cache the file on the local instance. Please do not use the file storage for data with strong consistency requirements.