The Great MP3 Bitrate Experiment

June 21, 2012

Lately I've been trying to rid my life of as many physical artifacts as possible. I'm with Merlin Mann on CDs:

Can't believe how quickly CDs went from something I hate storing to something I hate buying to something I hate merely existing.

Although I'd extend that line of thinking to DVDs as well. The death of physical media has some definite downsides, but after owning certain movies once on VHS, then on DVD, and finally on Blu-Ray, I think I'm now at peace with the idea of not owning any physical media ever again, if I can help it.

My current strategy of wishing my physical media collection into a cornfield involves shipping all our DVDs to Second Spin via media mail, and paying our nephew $1 per CD to rip our CD collection using Exact Audio Copy and LAME as a summer project. The point of this exercise is absolutely not piracy; I have no interest in keeping both digital and physical copies of the media I paid for the privilege of owningtemporarily licensing. Note that I didn't bother ripping any of the DVDs because I hardly ever watched them; mostly they just collected dust. But I continue to love music and listen to my music collection on a daily basis. I'll donate all the ripped CDs to some charity or library, and if I can't pull that off, I'll just destroy them outright. Stupid atoms!

CDs, unlike DVDs or even Blu-Rays, are considered reference quality. That is, the uncompressed digital audio data contained on a CD is a nearly perfect representation of the original studio master, for most reasonable people's interpretation of "perfect", at least back in 1980. So if you paid for a CD, you might be worried that ripping it to a compressed digital audio format would result in an inferior listening experience.

I'm not exactly an audiophile, but I like to think I have pretty good ears. I've recommended buying $200+ headphones and headphone amps for quite a while now. By the way: still a good investment! Go do it! Anyhow, previous research and my own experiments led me to write Getting the Best Bang for Your Byte seven years ago. I concluded that nobody could really hear the difference between a raw CD track and an MP3 using a decent encoder at a variable bit rate averaging around 160kbps. Any bit rate higher than that was just wasting space on your device and your bandwidth for no rational reason. So-called "high resolution audio" was recently thoroughly debunked for very similar reasons.

Articles last month revealed that musician Neil Young and Apple's Steve Jobs discussed offering digital music downloads of 'uncompromised studio quality'. Much of the press and user commentary was particularly enthusiastic about the prospect of uncompressed 24 bit 192kHz downloads. 24/192 featured prominently in my own conversations with Mr. Young's group several months ago.

Unfortunately, there is no point to distributing music in 24-bit/192kHz format. Its playback fidelity is slightly inferior to 16/44.1 or 16/48, and it takes up 6 times the space.

There are a few real problems with the audio quality and 'experience' of digitally distributed music today. 24/192 solves none of them. While everyone fixates on 24/192 as a magic bullet, we're not going to see any actual improvement.

The authors of LAME must have agreed with me, because the typical, standard, recommended, default way of encoding any old audio input to MP3 …

lame --preset standard "cd-track-raw.wav" "cd-track-encoded.mp3"

… now produces variable bit rate MP3 tracks at a bitrate of around 192kbps on average.

Encspot-omigod-disc-3

(Going down one level to the "medium" preset produces nearly exactly 160kbps average, my 2005 recommendation on the nose.)

Encoders have only gotten better since the good old days of 2005. Given the many orders of magnitude improvement in performance and storage since then, I'm totally comfortable with throwing an additional 32kbps in there, going from 160kbps average to 192kbps average just to be totally safe. That's still a miniscule file size compared to the enormous amount of data required for mythical, aurally perfect raw audio. For a particular 4 minute and 56 second music track, that'd be:

Uncompressed raw CD format51 mb
Lossless FLAC compression36 mb
LAME insane encoded MP3 (320kbps)11.6 mb
LAME standard encoded MP3 (192kbps avg)7.1 mb

Ripping to uncompressed audio is a non-starter. I don't care how much of an ultra audio quality nerd you are, spending 7× or 5× the bandwidth and storage for completely inaudible "quality" improvements is a dagger directly in the heart of this efficiency-loving nerd, at least. Maybe if you're planning to do a lot of remixing and manipulation it might make sense to retain the raw source audio, but for typical listening, never.

The difference between the 320kbps track and the 192kbps track is more rational to argue about. But it's still 1.6 times the size. Yes, we have tons more bandwidth and storage and power today, but storage space on your mobile device will never be free, nor will bandwidth or storage in the cloud, where I think most of this stuff should ultimately reside. And all other things being equal, wouldn't you rather be able to fit 200 songs on your device instead of 100? Wouldn't you rather be able to download 10 tracks in the same time instead of 5? Efficiency, that's where it's at. Particularly when people with dog's ears wouldn't even be able to hear the difference.

But Wait, I Have Dog Ears

Of course you do. On the Internet, nobody knows you're a dog. Personally, I think you're a human being full of crap, but let's drop some science on this and see if you can prove it.

On-the-internet-nobody-knows-youre-a-dog

When someone tells me "Dudes, come on, let's steer clear of the worst song ever written!", I say challenge accepted. Behold The Great MP3 Bitrate Experiment!

As proposed on our very own Audio and Video Production Stack Exchange, we're going to do a blind test of the same 2 minute excerpt of a particular rock audio track at a few different bitrates, ranging from 128kbps CBR MP3 all the way up to raw uncompressed CD audio. Each sample was encoded (if necessary), then exported to WAV so they all have the same file size. Can you tell the difference between any of these audio samples using just your ears?

1. Listen to each two minute audio sample

(update: experiment concluded; links removed.)

Limburger
Cheddar
Gouda
Brie
Feta

2. Rate each sample for encoding quality

Once you've given each audio sample a listen – with only your ears please, not analysis softwarefill out this brief form and rate each audio sample from 1 to 5 on encoding quality, where one represents worst and five represents flawless.

Yes, it would be better to use a variety of different audio samples, like SoundExpert does, but I don't have time to do that. Anyway, if the difference in encoding bitrate quality is as profound as certain vocal elements of the community would have you believe it is, that difference should be audible in any music track. To those who might argue that I am trolling audiophiles into listening to one of the worst-slash-best rock songs of all time … over and over and over … to prove a point … I say, how dare you impugn my honor in this manner, sir. How dare you!

I wasn't comfortable making my generous TypePad hosts suffer through the bandwidth demands of multiple 16 megabyte audio samples, so this was a fun opportunity to exercise my long dormant Amazon S3 account, and test out Amazon's on-demand CloudFront CDN. I hope I'm not rubbing any copyright holders the wrong way with this test; I just used a song excerpt for science, man! I'll pull the files entirely after a few weeks just to be sure.

You'll get no argument from me that the old standby of 128kbps constant bit rate encoding is not adequate for most music, even today, and you should be able to hear that in this test. But I also maintain that virtually nobody can reliably tell the difference between a 160kbps variable bit rate MP3 and the raw CD audio, much less 192kbps. If you'd like to prove me wrong, this is your big chance. Like the announcer in Smash TV, I say good luck – you're gonna need it.

So which is it – are you a dog or a man? Give the samples a listen, then rate them. I'll post the results of this experiment in a few days.

[advertisement] Hiring developers? Post your open positions with Stack Overflow Careers and reach over 20MM awesome devs already on Stack Overflow. Create your satisfaction-guaranteed job listing today!
Posted by Jeff Atwood
144 Comments

CD audio does not represent the quality of the studio master. You should definitely be prepared to buy that music again through iTunes, which has been storing 24/96 masters for a while now in preparation for sales to consumers. CD is 16/44 — it is less than half of the audio data in the studio master. You are listening to a shadow of the original music. Apple is collecting the masters so they can sell you the real thing — finally — and give you a reason to buy compared to streaming or YouTube.

MP3 is obsolete for 10 years now, and was replaced with AAC, which always sounds better at the same bit rate, is better standardized across players, and has better licensing. MP3 is part of MPEG-2, while AAC is part of MPEG-4. MP3 will disappear from your players first also, because of the licensing and how rare the files are becoming. Most of the world's perceptually-encoded audio is AAC, not MP3. And MP3 has wicked distortion and will cut off your whole high-end. Both bugs were fixed in AAC. There is not really any reason to compare MP3 encoders because none sound good. If you care about audio quality, you use AAC. That is what it is for. If you are nostalgic and have to use MP3, 128 kbits is the smallest you should use, because below that is a different encoder — even more primitive, even less suitable for music.

The best CD rip is a lossless PCM rip with error correction, and the CD should be cleaned first to reduce read errors, which in Red Book are not re-read, the player instead just makes data up to fill gaps. For a jukebox mix, do “iTunes Plus” (256 kbits AAC) because that sounds almost exactly the same as the lossless rip, but is small in file size and is the single most universally-playable audio file because no player ships that can't play what iTunes has been selling DRM-free for years now, and because AAC is ISO MPEG-4 the consumer audio playback standard for 10 years now, embedded in all mobile and PC hardware. If you also want a thumbnail rip, the smallest possible, AAC at 64 kbits will maintain the full 16/44 soundscape and sounds surprisingly good.

me.yahoo.com/a/LdFNQccxr8ALhp4J7kf7vop_B1cLurFc0YO3G28jRA-- on June 24, 2012 5:19 AM

I'm just going to be that guy and say that CDs are not exactly the highest standard themselves - vinyl is. It's sort of difficult to tell the difference between vinyl and a raw vinyl rip, but it's generally easy to tell the difference between either of those and a CD.

Not that that's very practical, or that it matters much if you're not using quality headphones (which I don't do myself on a day-to-day basis).

Sarah Wiseman on June 24, 2012 11:46 AM

I listened to all of them intently. I enjoyed all of them and wouldn't notice the quality disparity in any. I'll continue to live in blissful audio ignorance. I hope I never gain the competence necessary to be able to rank these in order.

churchskiz on June 24, 2012 7:38 PM

As a normal person, I just use Spotify. Problem solved.

128kbps is definitely not OK, but after 192kbps, I couldn't tell the difference really. And c'mon, just listen to the damn music.

Kamran Ayub on June 24, 2012 8:38 PM

For what it's worth, I encode all my music into THREE formats (thanks to Max which makes it trivial to do on Mac OS X): FLAC, Apple Lossless and 320kbps MP3. Apple Lossless is in addition to FLAC because unfortunately some FLAC files from years ago are no longer readable with the newest FLAC codec. Storage? I have plenty.

Dmitriy on June 24, 2012 8:52 PM

I generally consider myself to have pretty good hearing - I am a musician and sell recording gear for a living. I listen through a nice recording interface (well, reasonably nice) and $500 (each) reference monitors. I could only really hear a difference when I popped on the $200 headphones. I am a little surprised, though I suppose it depends on the song. Most songs I can easily tell if it's 128, but if it's 256 or better there's not much difference. Uncompressed just seems clearer, but I can't really place the difference there. All the in betweens are a wash for me. To those who say it is silly to use a song with crappy drum machines - there is a difference between lofi analog and lofi digital. We are talking about digital compression artifacts here and the drum machines don't have those.

Njglover on June 24, 2012 9:42 PM

I note that at least one person in the comments doesn't understand what lossless means, he said that flac sounds "almost as good as" uncompressed.

To that person, think of flac as a zip for music, data goes in, exact same data (bit-for-bit identical, hence lossless) comes out.

So it doesn't sounds "almost as good", it sounds exactly the same.

Randy Thiemann on June 25, 2012 12:28 AM

I'll try to understand what you do in this blog and continue discovering your records that gives me great pleasure to see it Because you have a real encyclopédie in your blog especially in this article.
Medium serieux

lora on June 25, 2012 1:29 AM

I enjoyed this test a little too much.

It seems that too many years of listening to loud rock has dulled my hearing ability. I used to be able to tell the difference easily, by listening to high-pitched stuff (such as cymbal crashes) but no longer. They all sound the same to me. ;( Right, where did those cassettes go to...

Damian Powell on June 25, 2012 4:11 AM

I look forward to taking the test being an audiophile person, at least to my friends and family. I enjoy good sound, that is all.

I have a few reservations to this blog post however and those are one, do we really need to be so effecient when it comes to space these days? and two, am I the only one who actually likes browsing my CD collection and enjoy taking the physical thing out of the box and inserting it into the machine? I love the experience of really listening to an album!

SharpAxen on June 25, 2012 6:56 AM

It's interesting to see what some believe are the answers. This was a test for distinguishing compression, not overall sound taste. I recall an older test that was similar to this and many people actually rated the highly compressed files as sounding perceivably better than lossless. I've trained myself to recognise the artefacts created by compression and when I listened on some really bad Logitech computer speakers (I'm too lazy to get out my good phones) I could still definitely tell the difference. I analysed the tracks (only after voting of course, I just couldn't wait for the results!) and found that I was correct except I had the 2 worst reversed. It will be interesting to see the success rate from the poll.

Mark Furneaux on June 25, 2012 9:09 AM

A few points...

a) if you're listening to these samples through your browser
b) if you're listening to these samples via the Windows WDM mixer
c) don't have your output sound device locked to the same output bitrate of the audio samples e.g. via ASIO

then you are not listening to bit-perfect audio, you are listening to interpolated (dithered) audio and all your audiophile arguments go out the window.

C,G,B,L,F :)

Dynamik on June 25, 2012 9:24 AM

One more vote for "they all sound the same". Or close enough to the same for me not to care, at any rate. Maybe if it was a song I liked...!

Matthew Marcus on June 25, 2012 9:58 AM

Jeff cheated by using a poorly mastered recording. Notice the clicks in the beggining? If he used something from ECM, I would INSTANTLY tell the difference.

kinokijuf on June 25, 2012 10:08 AM

Hacker News had an article about this not so long ago:
http://people.xiph.org/~xiphmont/demo/neil-young.html

The short of it is you don't need more that 44.1KHz/16 bit, uncompressed. Compression - well, that depends - lossy or loss-less, and what encoder.

Once question I'd ask it what is the objective? Good music, or perfect reproduction? ATRAC compressions was lossy, and two thirds of people could tell the difference - but two thirds of them preferred the ATRAC compressed version. I was very surprised by that when our lecturer told us that.

Andy on June 25, 2012 10:29 AM

Difficult. I can detect differences between them. I remember I conducted a test between two files ack when choosing the bit rate to use for my library. I was comparing kasbians club foot between 192 and 128, and all of a sudden, click- I could hear the waterynessif the cymbals,

It is a lot more difficult comparing between 5 samples especially on iPad ( with earphones of course). I think they are in descending order- I think the tell tale sign is the cymbal crescendo around 1:12. It's damn difficult- there is a difference- I could tell between two directly, maybe three, but five is just confusing me- I can't hear enough slight difference to rank them!

Hesitantant on June 25, 2012 12:34 PM

FLAC or Lossless WMA for storage, and CBR0 MP3 for playback.

AAC is not useful because most car stereos can't play it back (except if you have the stereo controlling an iPod/iPhone in head mode in which case the external device does the actual playback) and many portable devices can only handle AAC in LC (Low Complexity) mode.

It's unfortunate that most d/l vendors don't offer a lossless version, but I must applaud Hyperion Records, Qobuz store and a lot of artists on Bandcamp for giving FLAC/WAV options.

Mikenbondi on June 25, 2012 12:42 PM

I think that the second point is the one that everyone. Is missing. While it's nice to talk about how you are not using that much more space with something like flac, I have limited days on my phones data on my phone (2 gb per month currently) . Given the the cell networks current pricing scheme and business practice schemes (ie they're never going to change that cash cow) every byte I stream over my phone now becomes.precious. so as media companies move "to the cloud " (as I can't imagine them doing anything else to stay on business) that means that I won't actually be able to have the luxury of local storage. Whether you like it or not, that's where we're all headed. That makes this a valid argument.

Jon Velapoldi on June 25, 2012 1:15 PM

Jeff, why not use Ogg Vorbis instead of MP3 for lossy compression of your music collection? It's a free, non proprietary standard, (so better for archival purposes) and modern Vorbis encoders offer higher quality at lower bitrates than MP3. With the AoTuV fork of the encoder, you would probably be fine encoding your collection at 128 or even 96 kbps.

Miles Rotaru on June 25, 2012 3:00 PM

Pauli said: MP3 = LOSS at any bitrate. 320 kbps is great for earbuds - that's it. I will not waste time comparing cds to mp3. There is none. Buy vinyl and a decent table if you want to hear the recorded track in it's purest form. Until you jarheads start MAKING digital music (and some crap I hear these days sounds like 2 notes over and over... 010101010) you will always need to convert to analog to hear it.

Uh huh. Purest form. Right.

Well, except for the noise, and who cares about wear?

Tell you what, how about we get a CD and Vinyl of the same thing, decently mastered*, and compare output waveforms, or do some ABX testing (with "fake record noise" added to the CD so it's as dirty as a record is in the real world).

Vinyl is not pure - it's just distortion you're used to and prefer.

Then you can posture to me about vinyl after you pass the ABX test.

Problem is, audiophiles have historically been very bad at that:
http://www.bostonaudiosociety.org/bas_speaker/abx_testing2.htm
http://www.hydrogenaudio.org/forums/index.php?act=ST&f=21&t=7953


(* In fairness, some things these days are mastered with so much compression that they just suck, and the one indirect advantage vinyl has is that people mastering for vinyl never ever do that, but that's irrelevant to the format itself.)

Sigivald on June 25, 2012 3:02 PM

@Miles: Why create a lossy archive in OGG when almost every time you have to play it on another device it would have to be transcoded? It's even less useful than AAC in that respect.

Mikenbondi on June 25, 2012 4:45 PM

I think you stacked the deck alot with the choice of the clip you chose. I'm no audiophile, but I can hear the difference between MP3s, CDs, and records. For me it's a question of range. I can hear that the top and bottom are clipped. Maybe it's my age but I miss real bass - which you don't hear much of anymore.

Part of this is I feel that artists are producing for the medium that dominates (iToones, MP3s, etc.) MP3s don't sound much different from AM radio at times. The other part is folks can't tell the difference because they've never heard anything else - or not for a very long time or very often anymore.

I considered a number of songs to illustrate my points, but the one that won out was 'Rust Never Sleeps' by Neil Young. It's not that I'm such a big Young fan, but I can't think of a better example of extremes in a single record - the open base cords should move your furniture across the room (without cranking the volume too much)and Youngs voice should nearly set your teeth on edge. If they don't...well you're not getting the full experience.

Bob Singer on June 25, 2012 5:24 PM

@Adolfojp and @Gsuberland

I agree that you both makes a very good point here. However, I'd like to point out that the experiment is aimed to prove the "lossless vs lossy vs uncompressed" myth. We're looking for the best compressed format which, when played (= uncompressed on-the-fly ultimately to a speaker), gives the nearest-to-original quality. That is, INCLUDING the artifacts of old ! Tape hiss, low-bitrate drum samples, microphone clipping etc.

Although, I won't be surprised if some, or maybe even a lot, younger audiences who live in the "HD era" could not appreciate the imperfections of older recordings. But personally, when speaking about compression quality, I'd like the playback come as close as possible to the original.

A nice footnote I'd like to add: I piggyback this test for my personal purpose, namely to know whether I could perceive the difference between the 5 using various earphones, headphones, and speakers.

The conclusion:

A lot of difference are only beginning to be seen (heard) using HiFi or reference-class headphones or speakers. Namely, the lower end of the bass spectrum; the vocal definition; the upper frequency natural rings (cymbals, screams/high-pitched vocals); and the times when all the frequencies are mashed-up together.

Earphones do not help, btw.

In short: in the world of audio-listening, you want quality = prepare to burn a bit of a hole in your pocket. All the way in the audio reproduction chain:
- have a big harddrive, store your digital collection at least in FLAC.
- have a good player/soundcard
- have a decent amp
- have at least HiFi or reference-class speakers or headphones.

Then re-encode them in 160kbps' for your mobile phone/PMP speakers/earphones.

YumYumOfTheYum on June 25, 2012 5:46 PM

To my surprise, I discovered I'm no dog. Based on my post-analysis, my worst is actually #2 (where #1 is worst), and my best is actually #3.

However, when ripping from original source I'll go the archival route and rip to flac. Then I'll just re-encode for the device later, but most of the time I listen on my computer with fancy headphones anyway, so flac is used.

cellojoe on June 25, 2012 9:05 PM

The problem with your test is that you are using highly processed and compressed rock music that is filled with intentional distortion as artistic expression.

Over the years I have helped musicians design instruments, speakers, amplifiers and processing electronics to suit their needs. Invariably I am asked "what is the best kind" of speaker, amplifier, pickup, etc. My answer is always "whatever pleases you." The artist's choice, almost without exception, is something that significantly colors the sound with linear and non-linear distortion; after all, they are seeking something unique and expressive. While this is an excellent choice for performance, it is a terrible choice for reproduction! The responsibility of a performance system is to express the artist's vision, while the responsibility of a reproduction system is to render the original performance as faithfully as possible.

The recordings that you have posted for your test are reproductions and should be judged according to their faithfulness to the original performance, but how are we to do that? Do the recordings sound different? On my AKG headphones the answer is absolutely yes. Can I tell which is a more faithful reproduction of the original performance? The answer is absolutely no. In judging recordings of an electronic performance, there is simply no way to determine which reproduction is the most faithful without comparing it to the live performance.

This is why high quality acoustic recordings, be they rock, jazz, classical or folk, are more useful for judging reproduction quality. The reason is that our ears are already calibrated to what a guitar, xylophone, marimba, tabla or cello sound like in a live performance, which gives us a basis for comparison. Furthermore, a recording made in a natural acoustic environment allows us to judge whether the individual instruments and vocalists are precisely defined and well positioned on the stereo stage, or muddled together and muddy sounding (an artifact of non-linear distortions; the sum and difference frequencies referred to by another writer). This is simply not possible with studio recordings that have been electronically processed and mixed.

If your music library consists solely of electric rock and pop music then I agree that the differences between your samples are probably inconsequential. An electric guitar can have literally millions of different sonic personalities. How are we to know if it is reproduced correctly or not? But if you listen to well recorded acoustic music it is immediately obvious if a flute or violin or sitar "just doesn't sound quite right."

Davidesnyder on June 25, 2012 9:08 PM

I have perfect pitch and studied music from an early age. Using sennheiser hd headphones the difference between lossy (mp3 256 bitrate) and lossless (flac) is huge.

While doing music people couldn't clearly differentiate between the instruments on a track while using crappy headphones, after switching to good ones the instruments could be separated much more clearly by the ear. It is the same with lossy vs lossless.

The only way I can describe what is happening is the following: Have you ever been in the same room with a live drummer? Only you and him, to hear how he hits the bare metal? Now record him and listen to the difference. The lossless format is much much closer to the live sound. You hear it like the metal is hit before you. You do not need a trained ear to hear this difference.

Try to focus on only one particular instrument, see how easy it is in loseless vs lossy, hear it during the entire piece, do not let your mind fade from the guitar to the bass guitar for example. In loseless the instruments are 'clearer', not 'stepping on one another'.. It's easier to do this, just like they are playing in the same room with you.

Making the difference between 256kbps and 320kbps mp3 is much harder of course, even for a trained ear, but between 256 mp3 and flac, it's really obvious.

The experiment you are making is inconclusive, *at least* you should have people tested on the same headphone set. Even then it is difficult to judge any results because of different hearing finesse of the subjects.

You said "nobody could really hear the difference between a raw CD track and an MP3 using a decent encoder at a variable bit rate averaging around 160kbps". This is wrong.
If this was true, then we would be using 160kbps for high quality audio. We do not use lossless formats out of masochism, just for the computers to store the sounds without us being able to hear them.

So, in the end, if you can't hear a difference it doesn't mean others can't. Go to a recording studio near you and ask the guys there what they think.

Adi Ratiu on June 25, 2012 9:29 PM

Hello Mr. Horror. I have one problem with your experiment. You are only testing for accuracy and not for precision. It would be better if you had 3 songs with each song at 3 bitrates. This way you can measure whether people are judging wrong or their criteria is wrong. Personally I like songs encoded at 96 and think they are lossless. Don't like lossless CD quality and would rate it as 96.

Asytwu on June 25, 2012 10:17 PM

In striving to get rid of physical stuff, you tend to gather lots of data instead. Data that just gathers dust...

I'd go for Spotify instead when it comes to music. Stream to the phone or computer when you need it.

Superwiren on June 26, 2012 12:19 AM

If you're so concerned with efficiency, why not go with AAC rather than with MP3?

Julian on June 26, 2012 3:34 AM

The German computer magazine "c't" did an experiment back in 2000. They invited sound and music specialists (among them a famous German music producer and a blind person) into a studio and did a double-blind test to see if they could differentiate between MP3 and CD audio.

Bottom-line: "Our music-trained test listeners could tell the the poorer MP3 quality (128 kbit s) is quite accurately from the other two samples; between MP3, 256 kbit/s and the original CD however, they could not - on average over all the music pieces - tell the difference."

Because the first test created such an outrage among the readers of the magazine, c't released the samples and let the readers repeat the tests themselves. The result:

"The fact that some of the 128-kBit/s-recording [...] were consistently rated better than the originals from the CD, however, stunned even the involved editor …"

The quotes are taken from the follow-up article (c't 6/2000) on the original test article which appeared in c't 3/2000.

http://www.heise.de/ct/artikel/Kreuzverhoertest-287592.html

The original article is only available behind a paywall.

Little_endian on June 26, 2012 4:17 AM

@Julian: AAC is not efficient use of your time if you're deploying to portable units (radios, car stereos, DVD players etc) outside of the Apple ecosystem. WMA is the most commonly found compressed format after MP3 on most devices. Even expensive devices I've bought in the last year or two either don't support AAC at all, or require AAC-LC.

Mikenbondi on June 26, 2012 4:36 AM

It might be difficult to hear on your laptop's built-in audio card.

Lukaszlipnicki on June 26, 2012 6:14 AM

First off... GREAT EXPERIMENT.

Two key points I wanted to throw into the pile here...

1. Quality (bit rate) of the audio track is moot if you have shitty (apple headphone) output quality... you will indeed NEVER notice a difference. Audiophile grade recordings are only worthwhile and of value if the output (speakers) are reference grade. Believe me, if you love music as much as many of us do, you will indeed notice a difference.

2. Recording 192k FLAC from a CD is a true waste of space. You are not GAINING anything from the extended bit rate... since it has already been taken away when it was put down to "CD quality" bit rates. It's like drinking shite wine from crystal goblets... it's still going to taste like shite. Audiophiles will go to sites that sell original master recordings, RECORDED at 192k... and trust me, there is a massive difference. The voice isn't muddy, the bass is rich and punchy and the high end is clear and not brassy or sharp.

For those who are happy now, rock on...

Camwisegamgee on June 26, 2012 8:27 AM

Unfortunately, I'm just a normal guy... So I couldn't really hear the difference.

But I thank you for the song I didn't know and which was pretty good actually.

Peralma Jérémie on June 26, 2012 3:30 PM

Looks like the survey is closed, so I'll add my thoughts here.

Limburger and Feta both sound awful. I'm guessing Feta is the 128bkps one, and Limburger isn't too far off.

Brie is a bit better, although the compression artifacts are still noticeable on the sibilants.

Cheddar and Gouda are the tricky ones. I can't tell if any of them are compressed on their own, and comparing them against each other gets very subjective. My bet is on Gouda being the uncompressed one, as Cheddar sounds slightly duller in the high frequency domain.

I listened on AKG K271 mkIIs plugged straight into a MacBook Pro, no amp or external DAC. These are studio headphones with well defined upper highs, which is where MP3 compression is most noticeable.

I think anything above 256kbps is fine. I've settled on the V0 preset for my own library, harddrives are cheap.

Elektronaut on June 27, 2012 9:03 AM

It is quite hard. I think they should be ordered like this:

5 Feta = Worst
4 Gouda
3 Limburger
2 Cheddar
1 Brie = Best

Is it possible for you to list with the results the different kbps for each sample and the encoder use. I would also like to test it with other types of music, as other comments suggested.

Cheerio
Cartero33

Cartero33

D on June 27, 2012 9:18 AM

Three of them were so painfully bad that I had to stop after 10-15 seconds. One was acceptable, but not as good as the copy I have on my PC, while the other was clearly the genuine article. Then again, I do have sensitive ears.

The best policy, if you have the space and local law permits, to keep a reference copy of as high a quality as possible on your archive machine, then downgrade it according to the equipment available and likely listeners on the target device. For example, my desktop machine has the best quality I can get. This is often 192 kbps due to upstream decisions on the quality/download speed compromise or the music being just plain old, but I take 320 or lossless where possible. Even then I often prefer CDs because the CD-speaker connection is slightly cleaner than the hard drive-speaker route on my particular machine.

My PDA, on the other hand, uses an indifferent sound card to produce audio and has a bad speaker. The headphones I have are not much better because the good ones require a converter which reduces the sound to the point where the improved headphone quality is irrelevant. Music on that is either 8 kbps or 16 kbps, depending on which produces artefacts that mask the hardware imperfections better. For some really awkward tracks, I've even resorted to converting to MIDI, though that has major downsides (including losing the words sung).

Christine on June 27, 2012 9:38 AM

I must echo the complaints about the choice of material. Not only are their a lot of samples in there, which are low quality from the 80s, there is also a lot of digital reverb which has a smeary "digital sound. In addition, if this was "mastered" to CD in the 80s or 90s, a lot of those masters were done very poorly, and in general there is all sorts of processing applied during that stage. A lot of those early CDs sound downright horrible. Mastering techniques have improved a lot, but more recent masters of classic rock have to battle the decay of the original analog tapes.

So there is a vast range in "sound quality" of commercially released music. A good sounding recording at 128kbit mp3 will sound way better than a bad sounding recording at full bandwidth.

That said, I had a hard time hearing differences. One problem is that hearing subtle differences in recordings is hard. It takes critical listening skills. Also, remembering differences in sound if you have to go through a processing of switching sources that takes a few seconds is pretty much impossible. Our brains are constantly processing how we hear things, every time you listen to the same recording closely it will actually sound different!

I doubt anyone can rank the samples correctly in this case. On the other hand, in a good listening environment, I am sure I could rank the original against the 160kbit if I had a quick A/B comparison. At least I could tell a difference. Deciding which one sounds "better" is entirely subjective.

On Jeff's original point, well if he thinks encoding at 160kbit is good enough for his own listening, then it is!

B1daly on June 29, 2012 8:07 PM

Maybe of interest for some.

Comparing FLAC vs WAV: http://www.computeraudiophile.com/blogs/mitchco/flac-vs-wav-part-2-final-results-155/

Comparing 16/44 to 24/192: http://www.computeraudiophile.com/blogs/mitchco/16-44-vs-24-192-experiment-163/

And if you are a full-on audio/software geek and like playing with sound and DSP: http://www.computeraudiophile.com/blogs/mitchco/importance-timbre-sound-reproduction-systems-222/

Mitchatola on July 4, 2012 10:55 AM

Results please?

Jmtd on October 8, 2012 8:34 AM

theres a few rules when it comes to audio,

1. Audiophiles are insane and is basically a bunch of Placebo marketing bullshit, who like listening to their gear more than the actual music HeadFi forums have been known no ban any science, and one particular video I saw on youtube of a Headfier meet-up had some guy with a pair of $150 headphones hooked up to some Rackmount DSP-EQ and another rackmount Headphone amp, these electronics must be worth about $50k when yet he was using some seriously cheap headphones on it, and I'm sure this was his "portable" setup.

2. Placebo placebo placebo ( http://www.youtube.com/watch?v=G-lN8vWm3m0 )

3. Audio artifacts can be more audible on bad quality headphones as codecs are designed for a flat signal response and only the good quality reference ones actually have a good flat response (sites like Soundcloud and Bandcamp (128mp3) are actually far more enjoyable on my AGK Q701's compared to a cheap pair of headphones)

4. trust your ears, just make sure you remove the placebo effect and bias by doing blind listening tests, I was recently talking to some guy who thought 128kbps AAC encoded with lib_voaac sounded better than 128 mp3 encoded with LAME because his test was so far from reality that it was meaningless, if only he had used his ears he would have realised his testing method was stupid.

5. 128kbps is not equal to 128kbps, codec, encoder, and source all play a factor in quality thats why VBR is superior, and why all of youtube sounds like crap even though it using a superior codec to MP3, and another reason why you should buy lossless files so you can trust the MP3 is truly the best sounding one possible as you encoded it yourself.

6. Lossless is the 'better safe than sorry' option. mp3/vorbis/aac have some inherent mathematically flaws or just have buggy encoders that cause certain sounds "Killer samples" to artifact at any bitrate these usually don't appear in real world applications but can, so either blind test everything or be lazy and just use lossless ( http://www.hydrogenaudio.org/forums/index.php?showforum=35 ) and this allows you to convert to Vorbis, MP3 or AAC depending on what the device can play later.

Daniel Hill on October 10, 2012 6:31 AM

If you're insulting people who still have their hearing intact by calling them dogs, I'm sorry that you never learned true music appreciation. If you're playing on a computer, get a good sound card, get really good speakers, and compare 128kbps to 192kbps to compact discs. I've noticed a difference every time, since the 1990s. If you can't tell the different, your system is obviously sub par and you should consider upgrading hardware.

Yakimandu96 on November 13, 2012 6:04 PM

That is, the uncompressed digital audio data contained on a CD is a nearly perfect representation of the original studio master, for most reasonable people's interpretation moving to london

Jimmy Jam on November 13, 2012 11:59 PM

«Back

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been posted. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment