Appium 1.6 XCUITest reliability issues

[rg_software]

Hi all,

I managed to solve numerous troubles with Appium setup, and basically now I can run tests, but the system is extremely unstable. Would someone share their experience regarding the following issues:

1) Appium server is unstable. It often exits with the message:

[XCUITest] Launching WebDriverAgent on the device
[XCUITest] killing xcodebuild process: 0
Terminated: 15

Perhaps, the easiest way to reproduce this behavior is to abort tests without driver.quit(), and to restart them.
BTW, I wish there was an easy way at least to restart the server.

2) Device connection is unstable. Sometimes (to often...) when I start the next session the device (iPad) asks "Trust this computer?" question. Of course, I cannot just sit there and tap "yes" all the time It might be an issue of the iPad itself, but I must say that this problem is limited to Appium -- when I simply attach the device to my Mac, there are no questions of trust asked. Even if I say "yes", it does not help -- the application simply will not run. To run it, I have to restart both the server and the tests.

[pavanbachu]

Please mention more details like the logs and the device details and appium version

[rg_software]

OK, let's try.
1) Server stuff. I use Appium 1.6.0. Suppose I am in the middle of some test, then I abort it (exit Python script), then restart. The last things I see then is:

[MJSONWP] Calling AppiumDriver.createSession() with args: [{"appActivity":"com.unity3...
[Appium] Creating new XCUITestDriver session
[Appium] Capabilities:
[Appium]   appActivity: 'com.unity3d.player.UnityPlayerActivity'
[Appium]   platformName: 'iOS'
[Appium]   udid: 'bf91f5c4c32ce9b049d78a6aba9528f401d33c83'
[Appium]   deviceName: 'device'
[Appium]   appPackage: 'com.MyCompany.MyProject'
[Appium]   noReset: true
[Appium]   newCommandTimeout: 200
[Appium]   keychainPassword: '12345'
[Appium]   xcodeConfigFile: '/Users/me/Documents/appiumXcodeConfig.xcconfig'
[Appium]   realDeviceLogger: '/usr/local/lib/node_modules/deviceconsole/deviceconsole'
[Appium]   keychainPath: '/Users/me/Library/Keychains/MyKeychain.keychain'
[Appium]   app: '/Users/me/Downloads/MyProject_dev.ipa'
[Appium]   fullReset: true
[Appium]   platformVersion: '10.0'
[Appium]   automationName: 'XCUITest'
[debug] [XCUITest] XCUITestDriver version: 2.0.26
[BaseDriver] The following capabilities were provided, but are not recognized by appium: appActivity, appPackage.
[BaseDriver] Session created with session id: 411374a4-32c2-4ed4-944f-f0c2a0763fa4
[debug] [XCUITest] Xcode version set to '8.1'
[debug] [XCUITest] iOS SDK Version set to '10.1'
[debug] [XCUITest] Available devices: 59f7f8e3bd7f4801ab3e79132e7eaed089f7d5a2, bf91f5c4c32ce9b049d78a6aba9528f401d33c83
[debug] [XCUITest] Creating iDevice object with udid bf91f5c4c32ce9b049d78a6aba9528f401d33c83
[XCUITest] Determining device to run tests on: udid: 'bf91f5c4c32ce9b049d78a6aba9528f401d33c83', real device: true
[BaseDriver] Using local app '/Users/me/Downloads/MyProject_dev.ipa'
[debug] [BaseDriver] Copying local zip to tmp dir
[debug] [BaseDriver] Unzipping /var/folders/p0/50kwnybx093c6qfwpbqgm4vw0000gp/T/2016929-8482-198yzv7/appium-app.zip
[debug] [BaseDriver] Testing zip archive: /var/folders/p0/50kwnybx093c6qfwpbqgm4vw0000gp/T/2016929-8482-198yzv7/appium-app.zip
[BaseDriver] Unzipped local app to '/var/folders/p0/50kwnybx093c6qfwpbqgm4vw0000gp/T/2016929-8482-198yzv7/Payload/MyProject.app'
[debug] [XCUITest] Checking whether app is actually present
[debug] [ios-app-utils] Getting bundle ID from app
[debug] [iOSLog] Attempting iOS device log capture via libimobiledevice idevicesyslog
[debug] [iOSLog] Found idevicesyslog: '/usr/local/bin/idevicesyslog'
[XCUITest] Setting up real device
[debug] [XCUITest] Stdout from app isInstalled check: [....] Waiting for iOS device to be connected
Skipping J85AP 'iPad1' (59f7f8e3bd7f4801ab3e79132e7eaed089f7d5a2).
[....] Using iPad 3 'iPad2' (bf91f5c4c32ce9b049d78a6aba9528f401d33c83).
true

[debug] [XCUITest] App is installed.
[debug] [XCUITest] Full reset not requested. No need to install.
[XCUITest] Using default agent: /usr/local/lib/node_modules/appium/node_modules/appium-xcuitest-driver/WebDriverAgent/WebDriverAgent.xcodeproj
[XCUITest] Using default bootstrap: /usr/local/lib/node_modules/appium/node_modules/appium-xcuitest-driver/WebDriverAgent
[XCUITest] Launching WebDriverAgent on the device
[XCUITest] killing xcodebuild process: 0
Terminated: 15

2) Trust issues. I thought it appears on iOS 10, but now checked another device running 9.3.5, and I confirm the same: the dialog box "Trust this computer?" appears at some random moments, and the tests actually cannot be run even if I press "yes" (I need to restart Appium server). Apparently I am not the first to mention it: "you probably also notice that you need to trust the computer on each connection". Can someone comment on that comment?

I think it is related to XCUITest now, as I nad no such issues running Appium 1.5 with old UIAutomation.
Also I think in my case it is enough to restart Appium server and to disconnect/connect the iOS device to make this "trust?" dialog reappear.

[rg_software]

OK, in case someone faces the same troubles:

Trust issues happen due to restrictive permissions of lockdown. I managed to solve them by running
sudo chmod 777 /var/db/lockdown
before Appium. Note that these settings do not survive reboot, so you'll need to run it every time.

On reliability... Well, still the same. The only way to perform unattended testing for me is to continuously restart Appium server, unfortunately.

[pavanbachu]

@rg_software The best fix would be uninstall existing ideviceinstaller and ios_webkit_debugproxy and install them for master

brew install ideviceinstaller --HEAD

[rg_software]

Thank you. Do you refer to the trust issues or to the reliability of Appium?