Web Experimentationの評判・口コミ 全4件

What do you dislike?

Test results - the under the hood calculations could be more transparent, while the statistics engine does a ton of very complex calculations that we don't really need to see, in the end it provides one number for "statistical significance". As that number changes over time, sometimes its apparent fluctuations make no intuitive sense. This can make it difficult to justify test results to management and the rest of the team.

What do you like best?

A/B testing - this is the core functionality that Optimizely has always been great with and only got better as they add more functionality.

Test analytics - the reporting pages are great, the impact of the tests on revenue is extremely difficult to calculate correctly. Optimizely has made a lot of updates to their statistics engine, latest round of changes upset a lot of people, but it's still doing a great job presenting very complicated data in a simple way.

Personalization - still new and barely out of beta, but already looks promising and judging by the quality of the core product, A/B testing, this should be a hit with marketers as well.

Recommendations to others considering the product

This is true for any A/B testing platform - It's best used by someone who knows a bit of JavaScript, or has a testing team with members that have this expertise. While it's still entirely possible to make a lot of simpler "point and click" changes to your site early on using the built-in visual editor tools, as your testing program evolves some of the more advanced changes will start to require increasing amounts of HTML/JS?/Query knowledge.

What do you dislike?

Advance code has issues especially with conditional activation on heavy JavaScript frameworks.

Direct customer service via phone is not as transparent as it could be.

Experiments can take a while to load, and previewing is sometimes not accurate due to slow caching issues.

What do you like best?

Our organization was previously advancing different elements on our site based on design, or individual department preference. Now, using Optimizely, we are able to assess data and form a conclusion on decisions to change designs based on quantitative analyses. In addition, we have the flexibility to construct new products for our site within Optimizely's interface and deploy them without leveraging weight on our developers. Here are the points I love most:

Great for fast development especially with basic CSS components.

Transparent results allow results to be shared across all departments.

The user interface is very straightforward and Optimizely's free online documentation is an excellent resource.

Recommendations to others considering the product

Try to have a developer work with you when deciding.

What do you dislike?

The reporting is good, but could be better and more robust.

What do you like best?

Their user interface is ridiculously easy. Even if you don't code, you can use their interface. Their support team is awesome. They are so helpful and quick to answer questions.

Recommendations to others considering the product

If you want to improve your site. Use this company. Even if you don't know how to do something, contact their support and they are more than willing to point you in the right direction and help you achieve your goals.

Can't stress enough how easy their product is to use. To make a change on your site with their interface take so little effort is scary.

What do you dislike?

What I dislike, or rather what I wish for the product to have, is more of the unrealized potential.: for a consolidated end to end orchestration of tests across platforms.

For example, I'd like to see a test where a test starts on a mobile app and ends with conversion on a desktop site or mweb or vice versa, or also one where a multivariate test can be run across web with variations of app download ads and promos and resulting downloads and conversions. In other words a cross between referral marketing and Optimization or orchestrated events and goals across multiple platforms (web/mweb/mobile app) with consolidated progress, result and drill-down views on the Optimizely dashboard. At present these are tedious to set up and analyze (as one has to use a combination of referral tracking tools, analytics tools and Optimizely)

What do you like best?

I love the technical prowess of everyone I have met at Optimizely. The engineers and support folks have without exception, all been very smart and their customer service is really special. And I say this after dealing with many tech vendors .

Just as one example, when we first started out doing mobile integration, we experienced a few crashes, and they actually walked through code and debugger in live sessions with us, and the Opt-gineers (hey did I just invent a new term?) would spot something amiss, fix it in 10 to 20 minutes and send us a new .zip of the iOS SDK, and this would repeat until the problem was solved. They appeared in-exhaustable. They also seem never at a loss for an answer, a fix, a suggestion, all of which make complete sense and just work.

Also, I am truly impressed by the strength of the product. When you think about product architecture, and the end to end complexity you'd encounter if you ever attempted to implement a tool like this, from the backend infrastructure (which never appears to fail or show latency) to the intuitive interface for setting up, executing tests, viewing results, to the underlying algorithms and stats-engine at the heart of all of this (in itself the by-product of a Stanford PhD Thesis) the whole picture is one of some amazing engineering.

Recommendations to others considering the product

Highly recommend this product for desktop, mweb and mobile apps.