TestNest

A/B test app meta-data before the App Store release

1 follower

A/B test app meta-data before the App Store release

1 follower

TestNest helps A/B test and discover which icon, screenshots and description generates most app installs. Game publishing companies, app studios and individual app developers are spending huge marketing budgets to bring their app to attention among users.

Launch tags:Tech

Launch Team

Rise

Hunter

I'm very intrigued by the tagline of this product and I'm really curious how this works. @mrneek, can you give is some insights? EDIT: especially for Product Hunters @mrneek told me a little secret. Just press the 10px top right corner to get early acces.

Report

11yr ago

Product Hunt

@emieljanson Love this little secret. Will test TestNest ;)

Report

11yr ago

@emieljanson Love the secret access link as well!

Report

11yr ago

@emieljanson Also checkout Store Maven hunted earlier. They help A/B test your icon and app screenshots!

Report

11yr ago

Mogram

Maker

@kwdinc yep, Store Maven looks cool too. But from what I understood from their landing page the product is not yet ready and they basically provide concierge service for now instead. Kevin, maybe you have more info on this?

Report

11yr ago

Product Hunt

While clean, the site needs more information. How exactly are the a/b tests done? You can't do a true a/b test on the App Store itself.

Report

11yr ago

Mogram

Maker

@rrhoover hey Ryan, you are right. Recording a screencast with more information now. But in a nut shell - TestNest helps you create landing pages to test your app icon, screenshots etc. The landing page looks like this (icon test example): http://testnest.co/pages/53fb297... You direct traffic to it and get the analytics of the number of visits, conversions and CTR.

Report

11yr ago

Product Hunt

@mrneek gotcha! That was my assumption but it was unclear. This is useful although to @davidkmckinney's point, the testing meta-data/price on a custom landing page is a much different context than the App Store itself. Can users really depend on the results of TestNest to drive their App Store optimization or is it not designed for that use case?

Report

11yr ago

Mogram

Maker

@rrhoover that's a good point. We had a couple of interviews with various app owners that regularly do AB tests for their app metadata. They are using custom HTML pages for AB tests now and from their experience there is some difference in the results but it is not significant. That was enough for us to decide to go with custom landing pages for now and see how it is working out. But further down the road we are planning to add desktop templates that look like real app stores (but not 100% identical of course :) ). Same will go for mobile templates. And then we'll be able to compare the results of both and make a final decision.

Report

11yr ago

Castle

@rrhoover @mrneek Ryan I think it's more about the human condition that the APP Store Optimization which are two completely Different in the sense of context. One one hand this should be done first and then from using testnest you have a great control group for your weekly or bi weekly run on the app store. The focus is that you're always testing ASO as I always do I've launched 15 apps and several have done very very well and the #1 reason is because I do exactly what testnest does weekly but I did ti all manually. Also I can't stress this enough this helps with an international push. So a lotof times my control group is the US. I have to figure out what damange CTR am I forfeiting by say not localizing properly and localizing isn't just about changing words it goes much deeper. These guys have a chance to build something pretty significant here. The things I can do with TestNest...lord have mercy.

Report

11yr ago

Rise

Hunter

@rrhoover I agree with you on this Ryan.

Report

11yr ago

AirHelp

cool idea but how does one get traffic to it? If I buy ads I rather be testing the ad copy. If I post on facebook there will be too much noise from various relationships.

Report

11yr ago

Castle

@johnnyquachy The same way one would get traffic to any website/link...sharing the link in various forms but with A/B testing, one should be setting up where you'd want to test it...is it word of mouth, social media etc... it's about psychology/biology then anything else...it's about doing the testing to influence what you do on other platforms like Facebook Ads.

Report

11yr ago

Mogram

Maker

@johnnyquachy Hey Johnny, thanks for your comment. A lot of our users request a feature that will allow purchasing traffic right from their TestNest account, we are working on that now. Hopefully we'll be able to release it in the upcoming updates. For now we recommend to get traffic from google adwords or facebook ads (depending on the type of audience you are looking for). This way you will be able to get enough of _targeted_ traffic that will lead to _statistically_significant_ AB test results. If you only direct your facebook friends to the page or some random people that are not your target audience, you'll get inaccurate results. Or in other words your sample won't represent your app target customers. There are a lot of rules as well as best practices in AB testing that are available only to a limited group of people that do AB tests as their fulltime job. For example, here is a great tool to calculate a sample size for your AB test: http://www.evanmiller.org/ab-tes... You may want to also use A/A test (have two identical metadata elements in your test) to check whether you set everything up correctly (if A/A varian results will differ - something is wrong, as identical variants are getting different results), etc. Our goal is to eventually integrate all of those in TestNest and let all our users run AB test campaigns like pros :)

Report

11yr ago

@mrneek I think that's a must. My biggest trouble testing this is that I want to have a hassle free test. If I can quickly set-up a landing page, get traffic, and see results - I'm in.

Report

11yr ago

Mogram

Maker

@jasongurwin thanks Jason! Working on that already. Will keep you up to date when that feature becomes available.

Report

11yr ago

Product Hunt

@mrneek Tools that help developers market themselves in a better way are really needed, so this is good. I think perhaps you can give a small twist to your messaging... This is great for general A/B testing outside the App Store, but it's not about simulating the actual App Store environment or for App Store optimisation. So, a service for testing icons and landing pages is definitely useful. You could make it clear that TestNest is about testing *before* you get to the App Store optimisation stage. Then you have a clear value prop.

Report

11yr ago

Mogram

Maker

@davidkmckinney hey David, thanks for your comment. Can you please share your thoughts on the approach you think would be most effective here? We are still in MVP phase and very open for a great advice too! :)

Report

11yr ago

Lydia

@mrneek I think the second part "Run A/B test" is the unclear part. I was confused because the iPhone screen looks like an app store mockup. You should put a real example here I think. Mockups are a bit confusing!

Report

11yr ago

Mogram

Maker

@SoleneMa True, sorry about that. We'll probably update the mockups on the landing page until the time that we add mobile templates there. But as I already mentioned above further down the road we'll add templates that look like app page in the app stores.

Report

11yr ago

Mogram

Maker

@davidkmckinney ok, gotcha, thanks! So we'll update the messaging to make it clear. But our intention is to eventually be a tool for App Store optimisation. What would you suggest to change to get closer to that? Patch the desktop and mobile templates the way that it looks almost like app page of the app store?

Report

11yr ago

Castle

@mrneek That would be a "final" or 3rd say MVP yes that's what it should evolve to but focus right now on getting the little things right but yes. Also be mindful of iOS...if you test this out now say in a few weeks all of that is going to change when iOS 8 goes live...btw I also want both. When I'm testing I don't want something just in the app store. In Japan 90% of my downloads HAPPEN OUTSIDE THE APP STORE and on a landing page. I'm not your average user I've been A/B testing I'm a heavy user I can break the metrics down and get my cost low so I can spend pennies to convert etc I know what I'm doing and then you have your average user which will make up 70% probably of your userbase and even paying users don't forget about your heavy users!!! Please don't! Whenever you're in San Francisco Neek starbucks is on me! Just got funded 3 days ago man this is invaluable right now you have no idea!!!

Report

11yr ago

The Ultimate Guide To Medium (eBook)

Beautiful website. Did you get the flat icons and images from somewhere? What about the website template?

Report

11yr ago

Mogram

Maker

@gregmuender Thanks Greg. Took us few hours to make this website - true bootstrapped startup speed :) $12 for the template, about $5 more for a set of icons here http://themeforest.net/ + couple hours of photoshop and HTML editing.

Report

11yr ago

Mogram

Maker

@nicholassheriff thanks man. Hey, no problem, we'll add drag and drop and duplicate feature later this week.

Report

11yr ago

Castle

@mrneek Neek you're going to make me cry stop messing with my emotions like this. I was going to build this exact thing from scratch in a few weeks for my startup. Recently got funded for my web browser.

Report

11yr ago

Mogram

Maker

@nicholassheriff haha, sorry man :)) Just checked Quest - looks awesome! I love products that are visually appealing and easy to use. Congrats on the funding too!

Report

11yr ago

Castle

@mrneek Thanks man it's turning into something wonderful it's a long time coming. If you shoot me your UDID I can send you a pre release version. ( domaininfo@gmail.com )

Report

11yr ago

Mogram

Maker

Guys, need your advice. Here is a dilemma that we've recently noticed: on average one A/B test will need approximately 1000 visitors that represent your target audience (not just random people) to gain statistically significant results. Depending on the media buying skills and the keywords used, the cost of drawing such targeted traffic in google adwords or facebook ads would approximately be $0.5-5 per click * 1000 clicks = between $500 and $5000. That's quite affordable for marketing agencies, game studios and such. But for individual developers that's a lot of money. If we provide a simple tool to post google adwords then the efficiency would drop (since our users won't be able to target the traffic and optimise the ads as precise), so it may increase the cost too. So here is the dilemma - on one hand it is much better from the user standpoint to have all-in-one service with the ability to purchase high quality targeted traffic for your landing page. On the other hand it would be quite expensive and this may be a blocker for individual app developers in the end. What are your thoughts on this? Thanks!

Report

11yr ago

1 2