Finding an element inside android.webkit.WebView

[VicTheStranger]

Hi!

I am struggling with an issue of not being able to find an element inside android.webkit.WebView. In a nutshell, it is a ReactNative application that in a few places uses WebView content coming directly from the network. One such a place is user group management and the page in Android SDK uiautomator look like that

Since there are no resource id’s available, I though that the easiest way to find the input field (EditText) would be to start from the preceding text element “Employee number”. However, I could not do that with the code:

import com.microsoft.appcenter.appium.EnhancedAndroidDriver;
import com.microsoft.appcenter.appium.Factory;
import io.appium.java_client.MobileBy;
import io.appium.java_client.MobileElement;
import io.appium.java_client.remote.MobileCapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

public WebDriverWait wait;
public EnhancedAndroidDriver<MobileElement> driver;

@Before 
public void createAppiumDriverAndLogin() throws MalformedURLException, InterruptedException {
     DesiredCapabilities capabilities = new DesiredCapabilities();
    capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "android");
    capabilities.setCapability(MobileCapabilityType.NEW_COMMAND_TIMEOUT, 7913);
    capabilities.setCapability(MobileCapabilityType.AUTOMATION_NAME, "uiautomator2");
    capabilities.setCapability(MobileCapabilityType.SUPPORTS_LOCATION_CONTEXT, "true");
    capabilities.setCapability("autoGrantPermissions", "true");

    URL url = new URL("http://localhost:4723/wd/hub");

    driver = Factory.createAndroidDriver(url, capabilities);
    wait = new WebDriverWait(driver, 30, 500);
}

@Test 
public void updateGroupInfoTest() {
/* 
  Series of steps navigating through application menu to reach the desired page 
*/
driver.findElementByXPath("//android.view.View[@text=\"Employee number\"]/following-sibling::android.widget.EditText").setValue("12345");
}

When running the tests, I am getting “noSuchElement”. Trying to simplify the XPath first to

"//android.view.View[@text=\"Employee number\"]"

and then even to

"//*[@text=\"Employee number\"]

provides the Appium logs like that:

2018-07-03 16:58:49:608 - info: [HTTP] --> POST /wd/hub/session/0ba1d11a-0b58-49f2-8b0d-54cb8ecdf6e2/element {"using":"xpath","value":"//*[@text=\"Employee number\"]"}
2018-07-03 16:58:49:610 - info: [debug] [MJSONWP] Calling AppiumDriver.findElement() with args: ["xpath","//*[@text=\"Employee number\"]","0ba1d11a-0b58-49f2-8b0d-54cb8ecdf6e2"]
2018-07-03 16:58:49:611 - info: [debug] [BaseDriver] Valid locator strategies for this request: xpath, id, class name, accessibility id, -android uiautomator
2018-07-03 16:58:49:612 - info: [debug] [BaseDriver] Valid locator strategies for this request: xpath, id, class name, accessibility id, -android uiautomator
2018-07-03 16:58:49:612 - info: [debug] [BaseDriver] Waiting up to 0 ms for condition
2018-07-03 16:58:49:613 - info: [debug] [JSONWP Proxy] Proxying [POST /element] to [POST http://localhost:8200/wd/hub/session/e2d5fbaf-ee77-4501-ad91-1907349d3b5e/element] with body: {"strategy":"xpath","selector":"//*[@text=\"Employee number\"]","context":"","multiple":false}
2018-07-03 16:58:49:683 - info: [HTTP] <-- POST /wd/hub/session/0ba1d11a-0b58-49f2-8b0d-54cb8ecdf6e2/element 500 74 ms - 164

What am I possibly doing wrong?

I am using:

• Appium v1.7.1
• JDK 1.8.0.152
• io.appium:java-client:5.0.4
• org.seleniumhq.selenium:*:3.6.0

[mykola-mokhnach]

https://appium.io/docs/en/writing-running-appium/web/hybrid/

[VicTheStranger]

Thanks, @mykola-mokhnach!
I wonder why didn’t I see that article before… Anyway, it brought more questions, no real answers. When I check the contexts available on that application page, I see only one:

 Set<String> contextNames = driver.getContextHandles();
 System.out.println("context name: " + contextNames); 

context name: [NATIVE_APP]

Does that mean that the application does not have WebContentDebuggingEnabled? If so, why UI Automator can see the content inserted into native DOM, but Appium doesn’t?

[mykola-mokhnach]

Does that mean that the application does not have WebContentDebuggingEnabled

Yes, this is most likely the case

[VicTheStranger]

Still fighting with this one. After updating the application to enable WebContent as per https://stackoverflow.com/questions/44414913/debug-android-webviews-in-react-native, I can see the WebView in Chrome DetTools (chrome://instepct/#devices) and able to inspect the content of the downloaded page.

The Appium test can find 3 contexts on that application page: NATIVE_APP, WEBVIEW_com.my.app and WEBVIEW_com.my.app:lost. I believe the “duplication” of the WEBVIEW context is the ReactNative crap, so I had to make special precautions to pickup the name without “:lost”, i.e. WEBVIEW_com.my.app. However, switching the context as

driver.context("WEBVIEW_com.my.app");

is failing. The test console log contains only

Build info: version: '3.6.0', revision: '6fbf3ec767', time: '2017-09-27T15:28:36.4Z'
System info: host: 'executionhost19.prod', ip: 'fe80:0:0:0:36:d01b:9898:7421%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.13.3', java.version: '1.8.0_31'
Driver info: com.microsoft.appcenter.appium.AppCenterAndroidDriver
Capabilities [{appPackage=com.my.app, language=en, locale=en_US, deviceName=84B7N16506007796, deviceReadyTimeout=120, platform=ANDROID, appActivity=com.my.app.MainActivity, newCommandTimeout=7913, automationName=uiautomator2, locationContextEnabled=true, autoGrantPermissions=true, javascriptEnabled=true, udid=84B7N16506007796, platformName=ANDROID}]
Session ID: a2f345f7-ee27-42b7-8bbc-3ff7d3172bfb
    at com.my.app.test.AccountTest.updateVAGroupInfoTest(AccountTest.java:163)
Caused by: org.openqa.selenium.UnsupportedCommandException:
    <html>
    <head><title>401 Authorization Required</title></head>
    <body bgcolor="white">
    <center><h1>401 Authorization Required</h1></center>
    <hr><center>nginx/1.2.7</center>
    </body>
    </html>
Command duration or timeout: 600.11 seconds

and the Appium log indicates that appium-chromedriver was stuck for 10 mins and then was killed.The same 10 mins were spent when switching back to NATIVE_APP context:

    2018-07-05 10:18:59:090 - info: [HTTP] --> POST /wd/hub/session/a2f345f7-ee27-42b7-8bbc-3ff7d3172bfb/context {"name":"WEBVIEW_com.my.app"}
    2018-07-05 10:18:59:091 - info: [debug] [MJSONWP] Calling AppiumDriver.setContext() with args: ["WEBVIEW_com.my.app","a2f345f7-ee27-42b7-8bbc-3ff7d3172bfb"]
    2018-07-05 10:18:59:092 - info: [debug] [AndroidDriver] Getting a list of available webviews
    2018-07-05 10:18:59:092 - info: [debug] [ADB] Getting connected devices...
    2018-07-05 10:18:59:104 - info: [debug] [ADB] 1 device(s) connected
    2018-07-05 10:18:59:105 - info: [debug] [ADB] Running '/opt/android-tools-and-config/android-sdk-linux/platform-tools/adb' with args: ["-P",5037,"-s","84B7N16506007796","shell","cat","/proc/net/unix"]
    2018-07-05 10:18:59:210 - info: [debug] [AndroidDriver] WEBVIEW_9396 mapped to pid 9396
    2018-07-05 10:18:59:211 - info: [debug] [AndroidDriver] Getting process name for webview
    2018-07-05 10:18:59:212 - info: [debug] [ADB] Getting connected devices...
    2018-07-05 10:18:59:220 - info: [debug] [AndroidDriver] WEBVIEW_9141 mapped to pid 9141
    2018-07-05 10:18:59:220 - info: [debug] [AndroidDriver] Getting process name for webview
    2018-07-05 10:18:59:221 - info: [debug] [ADB] Getting connected devices...
    2018-07-05 10:18:59:228 - info: [debug] [ADB] 1 device(s) connected
    2018-07-05 10:18:59:229 - info: [debug] [ADB] Running '/opt/android-tools-and-config/android-sdk-linux/platform-tools/adb' with args: ["-P",5037,"-s","84B7N16506007796","shell","ps"]
    2018-07-05 10:18:59:241 - info: [debug] [ADB] 1 device(s) connected
    2018-07-05 10:18:59:242 - info: [debug] [ADB] Running '/opt/android-tools-and-config/android-sdk-linux/platform-tools/adb' with args: ["-P",5037,"-s","84B7N16506007796","shell","ps"]
    2018-07-05 10:18:59:404 - info: [debug] [AndroidDriver] Parsed pid: '9141' pkg: 'com.my.app' from
    2018-07-05 10:18:59:404 - info: [debug] [AndroidDriver]     USER      PID   PPID  VSIZE  RSS   WCHAN              PC  NAME
    2018-07-05 10:18:59:405 - info: [debug] [AndroidDriver]     u0_a5351  9141  529   1447708 256692 SyS_epoll_ 0000000000 S com.my.app
    2018-07-05 10:18:59:406 - info: [debug] [AndroidDriver] Returning process name: 'com.my.app'
    2018-07-05 10:18:59:410 - info: [debug] [AndroidDriver] Parsed pid: '9396' pkg: 'com.my.app:lost' from
    2018-07-05 10:18:59:410 - info: [debug] [AndroidDriver]     USER      PID   PPID  VSIZE  RSS   WCHAN              PC  NAME
    2018-07-05 10:18:59:411 - info: [debug] [AndroidDriver]     u0_a5351  9396  529   955208 68356 SyS_epoll_ 0000000000 S com.my.app:lost
    2018-07-05 10:18:59:411 - info: [debug] [AndroidDriver] Returning process name: 'com.my.app:lost'
    2018-07-05 10:18:59:412 - info: [debug] [AndroidDriver] Found webviews: ["WEBVIEW_com.my.app:lost","WEBVIEW_com.my.app"]
    2018-07-05 10:18:59:413 - info: [debug] [AndroidDriver] Available contexts: ["NATIVE_APP","WEBVIEW_com.my.app:lost","WEBVIEW_com.my.app"]
    2018-07-05 10:18:59:415 - info: [debug] [AndroidDriver] Connecting to chrome-backed webview context 'WEBVIEW_com.my.app'
    2018-07-05 10:18:59:425 - info: [debug] [AndroidDriver] A port was not given, using random port: 8000
    2018-07-05 10:18:59:427 - info: [debug] [Chromedriver] Changed state to 'starting'
    2018-07-05 10:18:59:437 - info: [Chromedriver] Set chromedriver binary as: /opt/appium/deps/appium/node_modules/appium-chromedriver/chromedriver/linux/chromedriver_32
    2018-07-05 10:18:59:439 - info: [debug] [Chromedriver] Killing any old chromedrivers, running: pkill -15 -f "/opt/appium/deps/appium/node_modules/appium-chromedriver/chromedriver/linux/chromedriver_32.*--port=8000"
    2018-07-05 10:18:59:459 - warn: [Chromedriver] No old chromedrivers seemed to exist
    2018-07-05 10:18:59:460 - info: [debug] [Chromedriver] Cleaning any old adb forwarded port socket connections
    2018-07-05 10:18:59:461 - info: [debug] [ADB] List forwarding ports
    2018-07-05 10:18:59:461 - info: [debug] [ADB] Running '/opt/android-tools-and-config/android-sdk-linux/platform-tools/adb' with args: ["-P",5037,"-s","84B7N16506007796","forward","--list"]
    2018-07-05 10:18:59:476 - info: [Chromedriver] Spawning chromedriver with: /opt/appium/deps/appium/node_modules/appium-chromedriver/chromedriver/linux/chromedriver_32 --url-base=wd/hub --port=8000 --adb-port=5037 --verbose
    2018-07-05 10:18:59:488 - error: [Chromedriver] Chromedriver exited unexpectedly with code 127, signal null
    2018-07-05 10:18:59:488 - info: [debug] [Chromedriver] Changed state to 'stopped'
    2018-07-05 10:28:59:092 - info: [HTTP] <-- POST /wd/hub/session/a2f345f7-ee27-42b7-8bbc-3ff7d3172bfb/context - - ms - -
    2018-07-05 10:28:59:116 - info: [HTTP] --> POST /wd/hub/session/a2f345f7-ee27-42b7-8bbc-3ff7d3172bfb/context {"name":"NATIVE_APP"}
    2018-07-05 10:28:59:117 - info: [debug] [MJSONWP] Calling AppiumDriver.setContext() with args: ["NATIVE_APP","a2f345f7-ee27-42b7-8bbc-3ff7d3172bfb"]
    2018-07-05 10:38:59:117 - info: [HTTP] <-- POST /wd/hub/session/a2f345f7-ee27-42b7-8bbc-3ff7d3172bfb/context - - ms - -

[Chandra306]

Hi Victor,
Could you able to resolve this issue? If yes, can you please share the details…I am facing same issue now

[VicTheStranger]

Hi,

Sorry, @Chandra306, cannot help you as I just got lucky with this one. In one of the following redesign cycles, the WebView frame got removed from the application I have been testing. I would still be interested to know what kind of trick did I miss, but the priority for it is obviously not that high.

Good luck!

Victor