Touch Actions and Mobile Web

That is correct! ccc

Hi,

I'm also facing this issue.
I'm running the tests on a real android device (galaxy tab3) and whenever I'm trying to use Touch Actions (e.g. driver.swipe() or driver.pinch()) I'm getting this:Not Yet Implemented error.
I'm using:
Selenium 2.42.2
Java-client 2.0.0
Appium 1.2.4

Device:Galaxy Tab3
OS: Android 4.4.2
Browser: Chrome 38

BTW:
I've tried switching the context to "NATIVE_APP" and yet it still doesn't work (I get a different error:An unknown server-side error occurred while processing the command).

Any Ideas?

Thanks in advance,
Daniel.

Definitely switch to the NATIVE_APP context, and look at the appium logs there. The unknown error could be something like trying to swipe out of bounds or something.

Hi Jonahss,

Thank you for the assistance!
Eventually I've been able to successfully execute TouchActions (swipe, tap etc..) by switching the driver to NATIVE_APP as you've suggested.

I now understand, that currently, appium supports Touch Actions only in native context (otherwise, I got a "Not implemented yet...error).
Unless I'm wrong, this means that if one desires to utilize the the benefits of TouchActions while testing a Mobile Web, he should get the coordinates of the WebElement he wants to manipulate via a TouchAction, switch to Native context, execute the touch command via an api that accepts X,Y coordinates and then, switch the driver back to the previous context like in the following pseudocode:

WebElement element = driver.findElement(By.id("testElement"));
Point point = element.getLocation();
Dimension size = element.getSize();

int elementCenterX = point.getX() + Math.round(size.getWidth() / 2);
int elementCenterY = point.getY() + Math.round(size.getHeight() / 2);

String originalContext = driver.getContext();
driver.context("NATIVE_APP");

driver.tap(1, elementCenterX , elementCenterY , 1000);

driver.context(originalContext );

Unfortunately, I still face problems trying to utilize TouchActions for mobile-web testing purposes.
The major problem, is the difference between the X,Y reference point that is used in "NATIVE-APP" context and the one that is used in "WEBIEW".
It looks like in "NATIVE-APP" mode, the driver regards the X,Y coordinates relatively to the physical screen in device pixels (screenXY), whilst in "WEBVIEW" mode, the driver regards the X,Y coordinates relatively to the to the top left of the fully rendered content area in the browser in CSS pixels (pageXY).

This means, that in-order to make things work, we need to somehow transform the aforementioned WebElement's PageXY coordinate to a screenXY coordinate.

I've tried several approaches to solve this issue - unfortunately without success.

Approach #1(use org.openqa.selenium.internal.Locatable to get the "onScreen" coordinate):
Point onScreen = ((Locatable)element).getCoordinates().onScreen();
Result: Fails - I get an error saying that AppiumDriver doesn't support the method: onScreen()

Approach #2 (Calculate the offset between the two reference points)
In order to do so, one of the basic things we need to do is getting the window size:
driver.manage().window().getSize();
Result: Fails -I get an error saying that AppiumDriver doesn't support window().

Would appreciate your kind advice!

Thanks again and Best Regards,
Daniel.

Thanks jonahss!

I've created an issue in the github as requested:

Thanks & Best Regards,
Daniel.

hi,
actually, I have faced this kind of error when i was using like e.g. tap(button).perform()
then I used (X,Y) co-ord but Y co-ord is not correctly located on screen while X co-ord is correct so i add some offset (in my case i add 100 i.e. Y+100) then its worked for me.
But i dont think this is the reliable way to do this job.

Hi guys!
Has anybody got it resolved? Please share your results as hardcoded Y offset couldn't be worked for different mobiles as they have different resolutions.

Eric knows another workaround, how to get this resolved.
But it also works unstable:-(

public static void tap(WebElement we) {
    Point p = we.getLocation();
    Dimension size = we.getSize();
    try {
      nativeContext();
      Point appLoc = driver.findElement(By.xpath("//UIAWebView[1]")).getLocation();
      driver.tap(1, appLoc.x + p.x + (size.width / 2), appLoc.y + p.y + (size.height / 2), 1);
    } finally {
      webContext();
    }
  }

Hi,

This doesn't work for me:

Point appLoc = driver.findElement(By.xpath("//UIAWebView[1]")).getLocation();

org.openqa.selenium.WebDriverException: An unknown server-side error occurred while processing the command.

Smth that worked on one phone. failed on another...
Is there any ideas how to get correct coordinates for tap?

@jonahss
Looks, this can be fixed in 1.3.2. When are you going to release 1.3.2?

It seems it has been fixed here:

and now is going with the new release.
Also, with the new server release this java client feature60 is going to be published.

Waiting for the 1.3.2

It's coming!
Running tests as we speak.

Hey, @jonahss, @SergeyTichomirov

Looks, this is no longer valid in 1.3.2:
executeScript("mobile: swipe", swipeObject)

Is there a way to tap, swipe and scrollTo element in webview?

Does it not work?
Have you tried

element.tap()
element.zoom()
element.pinch()
element.swipe(direction, int)

?