There’s a lot wrong with storage and memory system design these days. There are so many varying layers of caching and tiering that arbitrarily determining the best configuration is an unmanageable feat.
Multi-tiered system architectures can apply equally well to host memory and storage configurations, storage system design and content-delivery networks (CDNs). All of these types of systems are going to have ample options for the layering and connecting of the components.
Many design efforts to appropriately stack these layers amount to guesswork, expedience or financial motives. In the worst case a system designer will “guesstimate” the size and type of different layers based on a limited number of tests and a gut feel for characteristics of the workload. More commonly, an expedient approach is decided based on available resources. This could mean what’s on hand, what can be compatibly assembled readily, or what fits in the available amount of real estate. Finally, the decision is sometimes made by either side of the monetary coin, either in an effort to save costs for the consumer or increase revenue for the supplier.
Even when efforts are made to optimize such systems, they are often optimized for synthetic workloads. Standard benchmarks and load generators help optimize systems for their specific purpose, but aren’t representative of the way the systems are used.
Simulations to the rescue
This is a place where system architects could learn a lot from the hardware design industry. Chip and circuit designers rely on a multitude of simulations to ensure the configuration and working of circuits in silicon and on printed circuit boards. At the same time, an institutional fear exists of touching deeply embedded software implementation.
Simulations provide fast, inexpensive ways to determine sizing and connections in multi-tiered systems. Simulations can provide real-time, functional replication of a component in a memory and storage hierarchy. Many research projects, such as Ramulator and MQSim, tackle these for both memory and storage devices. They take into account device delays, contention and sequencing of events for devices, without having to experimentally determine values on genuine devices. This saves both the time of evaluation and the expense of having the hardware on hand.
Behavioral simulations provide the mechanism for joining and sequencing the series of events that happen in a system. They are focused more on the interaction of bits in a system than what each of those bits are doing. This is the place where understanding complex memory and storage systems becomes overwhelming. The variety of combinations increases exponentially with each layer. Being able to piece such a system together in simulation is a tremendous advantage over direct experimentation. The cost of gathering all the parts and the time of assembling them explains why experimentation at this level almost never happens.
Magnition saves the day!
Magnition provides a framework for rapidly building behavioral simulations. Input methods, decision points and simulated components can easily be pieced together in a variety of configurations to test design decisions. Multiple tiers stack up as deeply as needed, and the flavor and size of each tier can be varied in different simulation runs.
Magnition’s framework is built of functional simulation components available from a library, or customers can supply their own component simulations. These are rapidly configured together through a narrative.
Variability is introduced into the simulation by changing the size, speed or makeup of one or more components.
Magnition is designed to work with genuine IO traces, either gathered by the user or from Magnition’s library of scenarios. Instead of running synthetic benchmark workloads, why not have results that reflect a runtime environment?
Finally, Magnition has proven results. Working with a major enterprise vendor, Magnition increased the number of experiments per engineer per day from 2 to over 50,000. Before the engagement, only 50 variations of the environment had been tried. After working with Magnition, over 1,000,000 variations had been tested.
End the guesswork and fear involved in developing complex memory and storage systems. Relax, find out for certain if the design works. With Magnition.
Find out more at magnition.io or contact us directly at info@magnition.io.