Razvan.net

The place for apps & thoughts

Don’t trust the simulator, test on device

I think we all heard this advice, but it’s so much easier and convenient to just test in the simulator. Often it’s something we leave for later, but the longer we wait the bigger differences we may notice are.

There are things that gives no choice but testing on the device but there are things that work in simulator and does not on the device.

I just learned that while working on BadRain’s 1.2 update, so here’s my story…

For version 1.0 et 1.1 I was too lazy to buy a USB-C cable et link the AppleTV to the computer for testing on device. I used internal builds I tested on my AppleTV using TestFlight. The problem with that is that you can’t see the console logs or debugging informations.

So since then I bought another AppleTV and a USB-C cable and I was able to do the right thing.

By the way about the cable, don’t be cheap as I was, thinking a short one (0.49 feet = 15cm) will be enough. It’s really short and if your computer is on a stand is even worse.

Some things does not work on simulator and it’s not clearly said

I added an In App-Purchase in the game (I’ll talk about why when I’ll post the new version messages). I implemented the feature and then I tried to test it…

So here how I get the list of purchasable products:

SKProductsRequest calls the delegate method productsRequest if succeed

or request if fails

The problem was that I was always getting in the success method but with an empty response.products list.

I spend several days on that, trying to figure out what was wrong, redoing the same things in different ways, trying understand why in the tutorials I found about that they succeed but I fail to get that list.

And finally one day searching with different keywords leads me to the tvOS 9.0 Release Notes where it’s said:

Notes and Known Issues

StoreKit
Note
StoreKit (that is, In-App Purchase) does not work in Simulator.

So StoreKit works in Simulator for iOS apps since long time and tutorial were all for iOS, that explains why never saw someone saying it does not. It’s only in the tvOS simulator that does not (still non in tvOS 9.2).

That made me make the jump and order the USB-C cable and test the application on the device rather than just using the TestFlight build for real test on device.

It’s too bad Apple didn’t choose a better way to handle this kind of thing, like instead of succeeding the request and returning an empty list, failing it and giving a error message saying: “Not supported in the simulator”.

Some things work on simulator but not on device

In the app I use some .plist to save locally high score, achievements and in app-purchased items using the standard way:

Worked well in the simulator, and it was not easy to test for real on a device as I was already connected and get these infos form GameCenter or App Store. I noticed some strange things sometimes but not easy to spot the problem.

But a recent execution on the device I paid attention on console logs and saw:

After searching a bit I found a forum post talking about. So AppleTV apps don’t have data, just cache ou temp data that are deleted when app is ends. If you want something persistent you have to use iCloud or for not much infos UserDefaults.

So here I am, having to reimplement things that I thought worked (as they did in simulator) but in fact they don’t (as proved on device).

Conclusion

If some things don’t work on simulator and you think you’ve implemented them right, before spending hours trying to figure out what you’ve done wrong or searching if it’s a issue about that, test it on device see if it’s working.

If you can test almost everything on the device is good because that’s your target. Simulator helps you doing some things/tests faster but when you add a new feature make sure it works on the device and in the simulator so you’ll know how you’ll test it.

Happy coding!