Google, Bing and Babelfish: What’s the best translation engine?

by

Two months ago I wrote about an effort to evaluate the quality of the three major free machine translation (MT) engines:

  • Google Translate
  • Bing (Microsoft) Translator
  • Yahoo! Babelfish

Ethan Shen has wrapped up the project, soliciting input from more than 1,000 reviewers. He summed up his findings here.

Here are the findings that jumped out at me:

  • Google wins, hands down, translating longer text passages. No big surprise here.
  • Bing and Babelfish are competitive translating shorter texts (150 or fewer characters). Bing did quite well with Italian and German, while Babelfish did well with Chinese.
  • Google’s brand trumps all. About halfway through his test, Ethan removed the brand names from the search engines, so the reviewers did not know which engine was doing which translation. The change in results was significant. Reviewers were 21% more likely to say Google was better than Microsoft when they knew the brand names. And reviewers were 136% more likely to say Google was better than Babelfish.

This last finding is what poses the greatest hurdle for Microsoft and Yahoo!

When it comes to machine translation — perception is (almost) everything. If people think you’re the best translation engine, then you are the best.

Integration is the other key element of success, and Google Translate is doing well here also — I absolutely love the Chrome browser integration.

Ethan is not done with his research. This is only stage one. To help him with stage two, click here.

(Visited 1,038 times, 1 visits today)
John Yunker

John Yunker

John is co-founder of Byte Level Research and author of Think Outside the Country as well as 19 editions of The Web Globalization Report Card.
Tweet
Share

11 thoughts on “Google, Bing and Babelfish: What’s the best translation engine?”

  1. I have used google translate for spanish for some time and decided to try Bing. Bing is much more accurate especially with respect to word order and more common word usage. I still love google, but for Spanish, anyway, it seems Bing is currently the clear winner.

  3. Interesting research, but I am afraid that there may be too few responses for some conditions of language pair and text length, so that the results may not be statistically significant. I noticed that the table of results has many values of 25%, 33%, 50% and 75% which can be the case with two, three and four responses — hardly enough data to draw valid conclusions.

  4. Over the last few months I have noticed a significant improvement on Google Translate and I believe this is due to the “Do you have an a better translation” option they used to have. It is no longer there, however. Russian and French translations are so much better now. Babel Fish looks so old now by its use of literal translation. Haven’t used Bing.

Comments are closed.