A new approach to condensing data leads to a 99% compression rate

Xlabs website: http://xlabsai.com/

Saldana made an eight-second video of me talking into a camera. For a moment, let’s pretend that we’re in a court of law and call the video Exhibit A. The video, captured with the built-in camera on an ordinary Mac, consumed 1.83 MB in MP4 format, and Saldana stored the video on a USB drive that I happened to have brought with me. I expected him to compress and decompress the video immediately after making it, but instead, we took the video to a secure, off-site location. At this point, we were joined by XLABS Chief Executive Officer Oliver Christie.

To respect the XLABS group’s wishes for confidentiality, I won’t reveal anything about the off-site location. I’ll simply say that the amount of security surrounding the exact nature of the compression algorithm is impressive. In my naive view, I imagined that as soon as you discover a way to do 99% lossless compression, you can shout it to the world. If you’re worried about the process being stolen, mail it to yourself in a date-stamped envelope, or send it to a patent office where the algorithm, and the identity of its originators, is safe and sound. But that’s not the way it goes. At this early stage in the unveiling of the compression technique, XLABS takes great care to make sure that the algorithm doesn’t get into the wrong hands. They’re keeping the process under tight wraps while they refine the code, define the use cases, and secure financial backing. (While I waited for a short time in the secure area, I realized that I must be the poorest person to be seeing this demo. Others are probably potential backers with very deep pockets.)

The rest of the demo was fairly simple. On an ordinary Windows laptop, Saldana used the command line to invoke a utility that created a compressed file from my original video file. He performed this compression step twice, and the results were slightly different from one time to the next. The algorithm uses some encryption and the randomness used in encryption accounts in part for these differences. Saldana said that the algorithm’s own secret sauce plays a role in accounting for the differences.

1 Like

Hmmmmm… Find that hard to believe.

Would that make Google and Dropbox be able to offer 20 times more free space than now?

From their website

XLABS is using Artificial Intelligence to build the most advanced data compression software ever created

Video presentation here:

1 Like

Maybe they’re storing it in Pi?

Since Pi is a never repeating sequence (if you continue far enough), you can in theory store any block of data as just an offset (an index to where in Pi the sequence begins), plus the length of the block.

Decompression requires you fast forward along Pi to the offset, and then just read of the required number of bytes. Not easy, because the offsets and lengths of Pi needed will be very large indeed, but compression even harder!

However, if algorithms could be developed to solve this, it would give very high compression ratios.

3 Likes

If this is true it’s crazy but not surprising a computer figured out how to compress data and not a human. I’m stuck in 2015 and didn’t release how old it was.

If this is true, then that RITA neural network is a far bigger innovation than the compression algorithm.

I’m pretty skeptical though. That video is a year old and only has had 408 views and 2 likes. Looks like they’re not being taken seriously by the world so far.

It’s also pretty dangerous to roughly outline how you solved such a problem and then wait so long with filing patents for it. Once other research teams truly believe that a gem like that can be found with such a method, I’d imagine they’d be all over it to beat them to the punch with patent filing.

3 Likes

I’ve always thought if compression of that magnitude was ever figured out Pi would be involved.

They probably used something like this: http://cs.stanford.edu/people/eroberts/courses/soco/projects/neural-networks/Applications/imagecompression.html

Though that’s definitely lossy compression.

http://matt.might.net/articles/why-infinite-or-guaranteed-file-compression-is-impossible/

xlabs is probably a scam.

2 Likes

Certainly sounds like snake oil. I read a bit about them, and an interview with their founder and CEO Oliver Christie (who’s twitter says he’s an AI consultant)…that strangely was done by a furniture company! He says he’s an ex FX / Bond trader, and that Xlabs has four smart tech founders, him and Stephen in NY, and Marco and Jonas in France. The latter seem to have figured out the compression stuff.

When i read this interview with cofounder a few minutes ago, i concluded that this was snake oil, some of his answers to the questions asked are cringworthy.

1 Like

but you posted about it anyway

Well I didn’t read the interview with co-founder. Article i posted seemed some what believable and the writer maintained some good scepticism.

That’s him, seems to have a decent reputation aswell; Author of Java For Dummies, Android Application Development All-in-One For Dummies, and other books
https://twitter.com/allmycode

Also one has to wonder why they used only a very short video. Maybe Pi compression is being used, because the larger the file the longer it takes to find the starting point in Pi. 1MB might be easy enough if they found a fast method and 100MB might take years to find.

And the “random” could be more to do with “close” match rather than introduced randomness. This would allow them to be a lot quicker and a vid file can have changes without them appearing in the playback. But other data types would have to have exact reproduction.

THEN Why did they have a “reporter” into their super secret facility for a fund raising event. If you are trying to keep something so so so secret then you don’t tell your competitors that you even have such a thing. If your competitors don’t know you have something then they won’t try to steal it.

signs of snake oils marketing.

Maybe quantum computing could speed that process up exponentially.