[iOS9 UIAutomation] What is Appium approach to UIAutomation deprecation by Apple

Tree · Jan '16

I tried omitting the automationName: xcuitest from my capabilities and I can get further(also tried adding xcuitest to the desired-caps.js file inside the appium-base-driver package - with the same end result). It will launch a simulator and open my app. But as soon as the app is ready to interact with I get an error. It may be a problem with localhost on my machine but i'm not really sure). As in order to get the webdriveragent-driver to be accessible from the machine I run the tests on (im running the tests over the network), I had to change the default host address in index.js to my mac's ipaddress.

An excerpt of the error is as below.

'info WebDriverAgent Jan 13 12:28:42 AUK-PDMAC29 XCTStubApps[1136]: Built at Jan 13 2016 12:26:23
info WebDriverAgent 
info WebDriverAgent Jan 13 12:28:42 AUK-PDMAC29 XCTStubApps[1136]: WebDriverAgent started on port 8100
info WebDriverAgent 
info WebDriverAgent-driver detected that WebDriverAgent is running at url http://10.28.244.237:8100
info WebDriverAgent Jan 13 12:28:42 AUK-PDMAC29 XCTStubApps[1136]: ServerURLHere-        >http://10.28.244.237:8100<-ServerURLHere
info WebDriverAgent 
info WebDriverAgent-driver WebDriverAgent started at url http://10.28.244.237:8100
info JSONWP Proxy Proxying [POST /session] to [POST http://localhost:8100/session] with body:     {"desiredCapabilities":{"app":"/Users/Shared/Apps/FIPhone.app","bundleId":"com.fiserv.ubank"}}
info JSONWP Proxy Got response with status 200: {"value":{"capabilities":{"device":"iphone","browserName":"uBank Dev","sdkVersion":"9.1","CFBundleIdentifier":"com.fiserv.ubank"}},"sessionId":"DD388F24-9C24-4893-86FE-14560FE73D96","status":0}
info MJSONWP Responding to client with driver.createSession() result:    {"appPackage":"nz.co.mcom.phone","platformName":"iOS","autoAcceptAlerts":"true","app":"/Users/Shared/Apps/FIPhone.app","deviceType":"iPhone","devic...
info HTTP <-- POST /wd/hub/session 200 37075.029 ms - 261 
info HTTP --> POST /wd/hub/session/c6e07e2a-d427-4097-8616-09b875606d01/timeouts 
info MJSONWP Driver proxy active, passing request on via HTTP proxy
info JSONWP Proxy Proxying [POST /wd/hub/session/c6e07e2a-d427-4097-8616-09b875606d01/timeouts] to [POST http://localhost:8100/session/DD388F24-9C24-4893-86FE-14560FE73D96/timeouts] with body: {"type":"implicit","ms":3000}
info JSONWP Proxy Got response with status 200: {"value":"Unhandled endpoint: /session/DD388F24-9C24-4893-86FE-14560FE73D96/timeouts -- http://localhost:8100/ with parameters {\n    wildcards =     (\n        \"session/DD388F24-9C24-4893-86FE-14...
info JSONWP Proxy Replacing sessionId DD388F24-9C24-4893-86FE-14560FE73D96 with c6e07e2a-d427-4097-8616-09b875606d01
info HTTP <-- POST /wd/hub/session/c6e07e2a-d427-4097-8616-09b875606d01/timeouts 200 8.915 ms - 293`

pr4bh4sh · Jan '16

Hi @Tree Appium dev team will be able to help you much better if you could log this as an issue at git19.

jonahss · Jan '16

Hi @Tree thanks for helping us test xcuitest driver.

You don't need the 'automationName: xcuitest' if you're running the xcuitest driver by itself. That desired capability will only be added for the main Appium driver, so it can know which driver (regular ios or xcuitest) to proxy commands to.

I think your problem right now is that the xcuitest driver does not recognize the /timeouts endpoint and is responding with an error. The test should continue just fine, but your test script may be exiting when it hits the error after sending a timeout command.

Please respond with more bugs and bug reports by creating issues on the github repository for xcuitest. Appium discuss is usually for general discussion, and not for technical debugging.

appium · Jan '16

Hi @jonahss,

Your input is awesome.
We're working on building an iOS/Android automation framework and wanted to check with you on this. Our biggest concern is Apple deprecating Instruments which comprises UIAutomation framework. How does this all come thru with Appium? Even though the majority of the mobile methods we're using are the regular find methods, we also call into findElementByIOSUIAutomation method. So, we would have to rewrite all of our iOS test cases to use the new XCUITest framework and call them from Appium? or bypass Appium completely and execute the XCUITests thru XCode plugin from Jenkins CI?. We don't want to maintain 2 different code bases (XCUITest with iOS and the regular Appium/Selenium methods written in Java when working on Android).

Please advise us. Is there a timeframe on when Apple would deprecate Instruments for good?
Btw, the new XCUITest doesn't work with iOS 8 and below. So, there we are with another dilemma.

We're totally confused here and any clarifications would be great.

appium · Jan '16

Anybody have any pointers?

jonahss · Jan '16

Hi @appium

Yes you are correct that XCUITest will not work on versions of iOS prior to iOS 9. For older versions of iOS, users can still use Instruments. Keep your older versions of Xcode, so that once Apple removes Instruments support you will still be able to run tests.

We don't know Apple's timeframe at all It would be really useful for us all if we could get an answer from Apple. I will try sending them a message, but if you (and others) post bug reports to Apple asking about it maybe they will answer us.

findElementByIOSUIAutomation is something I hadn't thought about. You are right, with XCUITest, this locator strategy will not be available at all.
Hopefully the new API in XCUITest will be more reliable and you can just use the regular Appium findBy methods. If you come up with specific cases where you cannot locate an element, create an issue on the appium-xcuitest-driver repository on github and we can figure out how to solve the issue.

appium · Jan '16

Thanks @jonahss

With Appium 1.5 release imminent, will it support calling test cases written in objective-c/swift from XCode using XCUITest and return the results of each of the test case run back to Appium?

or, I'm curious to know more about the automationname: xcuitest or the new xcuitest driver. what is this all about?

Can you please explain?

Thanks for your wonderful support.

jonahss · Jan '16

Appium 1.5 will have no new features. It is a major architectual refactor of the code.

So Appium 1.5 will still use the same Instruments approach that appium 1.4.16 uses today.

After Appium 1.5, we will be able to add the appium-xcuitest-driver. At first it will be a 'beta' feature, so users will have to send automationName: xcuitest to use Xcuitest instead of Instruments. This could be added maybe in two months?

After we are sure that there are no bugs in xcuitest-driver and it has all the features of the old ios driver, we will switch the default driver for all iOS jobs to the xcuitest driver. This probably will not happen for about 3 or 4 months from today.

For now, users can install or clone the appium-xcuitest-driver and test it by itself.

The appium-xcuitest-driver uses the same Mobile JSON Wire Protocol that Appium and Selenium have always used. Your tests should not need to be changed at all except that findElementsByIOSUIAutomation will no longer be implemented. Right now there is no plan to add the ability to write custom tests in objective-c/swift. If there is a feature that needs to be implemented, please create an issue on the appium-xcuitest-driver repository on github.

appium · Jan '16

Thanks @jonahss

Going forward, instead of findElementByIOSUIAutomation, it would be great if you add support for calling into XCUITest test cases written with Objective-C/Swift and parse the results from within Appium supported languages (java, python etc). In this way, we get a huge benefit. The test cases would be very fast on iOS as we're using Objective-C/Swift to write the test cases in XCUITest framework. This is very similar to findElementByIOSUIAutomation call which accepts javascript code leveraging Apple UIAutomation framework.

Just my 2 cents.

jonahss · Jan '16

Yes, I understand that. Noting your request.

But this would require us to write a Swift parser and executor in Swift. Last time we were using javascript so we could call eval(). On Android, I had to write a recursive descent parser and use Java Reflection to create a way to dynamically run the Android UIAutomator selector. It was a Java compiler written in Java. This was fun, but I had to hard-code support for only a couple Java classes and functions.

The Webdriver api should really be able to provide the means to do everything you want to do with testing. Here at Appium, we want to create methods to perform standard test actions, and don't mean to give users way to execute arbitrary code.

If you want to run plain old XCUITests on an app, then I suggest using Xcode and just having a native XCUITest suite.

If you find something that you can do with XCUITest, and you can't find a way to do it with appium-xcuitest-driver (or our method is too slow), then let's look at each of these cases individually and see if we can create new methods to help

appium · Jan '16

Thanks @jonahss

I'm more interested in the appium-xcuitest driver. Where can I get more information on that and are there any samples in using that driver?

Btw, I was also cheking this space: https://forums.developer.apple.com/thread/1406332
Any progress on your request to Stefan Lesser? That will immensely help us in leveraging XCUITest alongside with Jenkins.

Thanks.

Venkatesh · Jan '16

HI @jonahss

You said Appium will continue using instruments, but instruments uses UIAutomation API for performing Automation.

How different will be XCUITest approach for Automation if UIAutomation gets deprecated.

senevit4 · Feb '16

HI @jonahss
Speaking about findElementByIOSUIAutomation will no longer be implemented: and what about Predicates element search strategy(described here https://github.com/appium/appium/issues/263919)? It will be no longer supported in the future version of Appium(Does XCUITest support Predicates at all?)? What element searching strategies will be supported after Instruments will become completely deprecated? Only Xpath and Id?

jonahss · Feb '16

No samples of xcuitest driver yet, it's pretty new. But you can npm install appium-xcuitest-driver and run it with node. Now it's an appium server and will run ios tests.

No progress on that thread, but Apple is encouraging us to send bug reports and feature requests, so send in a request for a way to run xcuitest alone from commandline without xcodebuild

jonahss · Feb '16

The Xcuitest approach is completely different under the hood, but has the same exact programming interface. So there should be no changes to your tests when we make the switch. This is the benefit of adhering to the JSON wire protocol.

jonahss · Feb '16

XPath, Id, ClassName are supported now.
Predicates are part of findElementsByIOSUIAutomation so won't be supported.

jablaze · Feb '16

Hi. Do you plan on having the transition to the UI Testing libraries finished within that 6-mo. window? (Actually 4 mo. at this point)

jonahss · Feb '16

Yeah it's already almost done. You can even try it out now.
It's missing a couple methods, and doesn't have real device support yet. But these aren't hard to add.

jablaze · Feb '16

That's good news. Which version is it in? We use both the Server app and the node server.

heliojr · Feb '16

Hey ,
I am trying to test the appium-xcuitest-driver, but I am getting the following error after the simulator is started.

info WebDriverAgent-driver Using default agent
info WebDriverAgent-driver Launching WebDriverAgent on the device
info WebDriverAgent-driver Waiting for WebDriverAgent to start on device
info WebDriverAgent-driver xcodebuild exited with code 65 and signal null
ERR! WebDriverAgent-driver Error: xcodebuild failed with code 65
ERR! WebDriverAgent-driver at SubProcess. (lib/webdriveragent.js:73:18)
ERR! WebDriverAgent-driver at emitTwo (events.js:100:13)
ERR! WebDriverAgent-driver at SubProcess.emit (events.js:185:7)
ERR! WebDriverAgent-driver at ChildProcess. (lib/teen_process.js:177:14)
ERR! WebDriverAgent-driver at emitTwo (events.js:100:13)
ERR! WebDriverAgent-driver at ChildProcess.emit (events.js:185:7)
ERR! WebDriverAgent-driver at Process.ChildProcess.handle.onexit (internal/childprocess.js:200:12)
ERR! WebDriverAgent-driver [Error: xcodebuild failed with code 65]
info WebDriverAgent-driver Shutting down WebDriverAgent
ERR! MJSONWP Encountered internal error running command: TypeError: Cannot set property 'sessionId' of undefined

any idea how to fix it ?

Thanks
Helio

share a link to this topic