Part 2 in our series on Data Problems & Data Compression

Part of our series on Software and Data Management

The Way Companies Manage Growing Data

A Cloud Marketing Study conducted by IBM found that 90 percent of the data on the Internet was created in the last two years (2016 – 2018)! It's no surprise that both individuals and companies alike are looking to data compression to simplify their data-sharing habits. For organizations that have a lot of databases, a compressed database be a significantly better solution than an uncompressed file. However, compression may not always be a flawless solution. 

The Pros
Data compression, whether it's for a single database or an entire library, can greatly improve your file management capabilities by turning many input files into one output file, increasing your data per footprint capacities while eliminating space-draining data redundancies. Of course, it will also help maximize valuable disk space and conserve bandwidth, the cost of which can quickly add up if you're constantly downloading data for clients. In addition, data compression can also help reduce both the direct and indirect costs associated with storage of data.

Using data compression, Amazon Redshift was able to reduce the size of its largest database by 30 percent, which translated to more disk space and significantly lower costs for its clients. 
If you have clients who regularly receive physical copies of their data, the time to produce and ship these disks will be greatly reduced.  
 
Client Support 
However, in order to make sure that your client can actually access the data that they need, you need to follow best practices. This includes knowing which file formats a user can support through their platform or OS so that the deliverable file archive is read/writable on the user's end. The most commonly supported file formats are 7Z, TAR, RAR and ZIP. 

 
The Cons 
You must also bear in mind that not every file type is well-suited to compression. This is especially true for multimedia content. Likewise, encrypted data can't be compressed at all, so clients who need this type of data will need an alternative solution. It's also key to find a balance between a high compression ratio, the computationally intensive algorithms that make it work, and an increased response or latency time. Ideally, you'll also want to have a fixed maximum time frame for data decompression so that it doesn't take too long for your client to access their data. 
 

Does it have to be this way?
Fortunately, many of the downsides of data compression can be resolved using the right solutions and expertise. Read our next article to learn about specific data compression solutions that have little or none of these typical downsides.