You have systems all over the country, collecting various image files over FTP, cropping, scaling and storing them with their metadata and querying all with a nice interface, maybe even have alerts for specific images.
- Images take up space that the current systems don’t have enough of.
- All that image cropping scaling takes valuable time and effort.
- Storing the processed images requires even more capacity.
Increasing the capacity for an existing system usually means a NAS. It is a widely known and a powerful concept, being able to use a filesystem stored away from the main machine. But this time, we took a different approach.
The existing product is actually designed for metadata processing, indexing and querying, including user interfaces and all. So why not take all the other parts away from it and let it do its job efficiently! Why not:
- set image sources to send images directly to a specialized device
- having large capacity with a FTP interface
- storing the images reliably
- sending only the metadata to the indexer
- serving images to the users directly over HTTP
- and doing the image processing on-the-fly
This makes up the basis for the image storage part of the solution.
But the original problem was that the required capacity was not being known beforehand. That’s possible, not all problems start with clear-cut requirements. And that means it’s also possible even for this solution to fill up earlier than expected. So we built-in the ability to:
- add more of these devices when necessary to increase the capacity
- form a cluster to easily manage more than one device
- keep using the same endpoints even after moving to a cluster
- automatically leveling the used capacity of all the devices
- balancing the image processing workload among them
- and keeping the service highly available by using other members in place of down ones
In the end this proved to be a very flexible and a low-cost image storing and processing solution. Currently it’s all over the country with hundreds of machines already deployed and new ones being added everyday.