Two months ago I wrote about an effort to evaluate the quality of the three major free machine translation (MT) engines:
- Google Translate
- Bing (Microsoft) Translator
- Yahoo! Babelfish
Ethan Shen has wrapped up the project, soliciting input from more than 1,000 reviewers. He summed up his findings here.
Here are the findings that jumped out at me:
- Google wins, hands down, translating longer text passages. No big surprise here.
- Bing and Babelfish are competitive translating shorter texts (150 or fewer characters). Bing did quite well with Italian and German, while Babelfish did well with Chinese.
- Google’s brand trumps all. About halfway through his test, Ethan removed the brand names from the search engines, so the reviewers did not know which engine was doing which translation. The change in results was significant. Reviewers were 21% more likely to say Google was better than Microsoft when they knew the brand names. And reviewers were 136% more likely to say Google was better than Babelfish.
This last finding is what poses the greatest hurdle for Microsoft and Yahoo!
When it comes to machine translation — perception is (almost) everything. If people think you’re the best translation engine, then you are the best.
Integration is the other key element of success, and Google Translate is doing well here also — I absolutely love the Chrome browser integration.
Ethan is not done with his research. This is only stage one. To help him with stage two, click here.
Tweet this post on Twitter
Subscribe to Global by Design
8 responses so far ↓
1 Building A Cross-cultural Web Design For A Wider Audience | Onextrapixel - Showcasing Web Treats Without A Hitch // Jun 22, 2010 at 11:01 am
[...] recent study found that Google Translate was the best over-all machine translation engine, while Bing and [...]
2 Building A Cross-Cultural Web Design For A Wider Audience « gabriel catalano | in-perfección // Jun 22, 2010 at 12:22 pm
[...] recent study found that Google Translate was the best over-all machine translation engine, while Bing and [...]
3 Building A Cross-Cultural Web Design For A Wider Audience | CSS Citadel // Jun 22, 2010 at 4:04 pm
[...] recent study found that Google Translate was the best over-all machine translation engine, while Bing and [...]
4 Building A Cross-Cultural Web Design For A Wider Audience | Greatest News & Informations // Jun 25, 2010 at 2:52 pm
[...] recent study found that Google Translate was the best over-all machine translation engine, while Bing and [...]
5 Miguel // Jul 4, 2010 at 2:47 pm
I have used google translate for spanish for some time and decided to try Bing. Bing is much more accurate especially with respect to word order and more common word usage. I still love google, but for Spanish, anyway, it seems Bing is currently the clear winner.
6 Building A Cross-Cultural Web Design For A Wider Audience | FireStone Magazine // Aug 2, 2010 at 3:59 am
[...] recent study found that Google Translate was the best over-all machine translation engine, while Bing and [...]
7 ucuz hoteller // Aug 12, 2010 at 3:08 pm
my vote for google forever :) but it couldn’t be correct translate senteces from English to Turkish.
8 Gregor Erbach // Aug 17, 2010 at 3:54 am
Interesting research, but I am afraid that there may be too few responses for some conditions of language pair and text length, so that the results may not be statistically significant. I noticed that the table of results has many values of 25%, 33%, 50% and 75% which can be the case with two, three and four responses — hardly enough data to draw valid conclusions.
Leave a Comment