AI solutions provider builds scalable shared-file storage system with AWS
London-based Faculty provides AI solutions to more than 2,000 data scientists worldwide, who use its platform to write code which allows them to access large datasets and build containerised ML (machine-learning) models.
Faculty’s platform has run on Amazon Web Services (AWS) since its launch in 2014. As the company grew, it required a shared-file storage system with greater scalability to support its larger customer base.
Chief Technology Officer Andrew Brookes said: “Some projects require up to 10 terabytes of storage, and we needed the data science infrastructure to be able to scale easily to meet that, without us needing to provision or resize the file system to accommodate variance in data size.”
The firm wanted to scale its file system without needing to spend time maintaining the its underlying infrastructure. Additionally, it was looking to provide users with an easier, more effective way to collaborate in a shared workspace.
To meet these challenges, Faculty turned to Amazon Elastic File System (Amazon EFS), which offered a scalable, cloud-native solution to its shared storage needs. All of the company’s data is now stored on a single Amazon EFS system, with one subdirectory per project.
The system offers users a shared workspace, which can be mounted onto their individual data science environment, with any changes to an ML model immediately logged on the machines of data scientists and their collaborators.
With Amazon EFS, Faculty has a reliable and secure data storage solution that means it no longer has to manage file storage for different projects. Migrating to Amazon EFS has also enabled Faculty’s engineers to focus more on innovation.
Amazon EFS’s elastic scalability is allowing Faculty to better support the data storage needs of its customers. Furthermore, it can now more quickly deploy its platform to new users due to its use of AWS CloudFormation to automate provisioning for its file system.
Ultimately, the migration to Amazon EFS has enabled Faculty to help improve collaboration between the data scientists that use its platform, helping it to deliver a better product for its customers.
