iOS ImageElement - Manually set image or device resolution (using Appium 1.9.1, python client)

[qa_adrian]

Appium v1.9.1
python-client 0.28

Hello, was wondering if there is a way to change the window size for iOS to use the true resolution of the device rather than a scaled version? OR have find_element_by_image() use a scaled version screenshot as a reference image.

Cheers,

BACKGROUND

I’m using the -image locator option but I can’t seem to get iOS to tap on the image after locating it (no issues with Android using the default settings)

Seems due to pixel scaling on iOS, the location tapped is off.

DEVICE INFO

iPhone 7 - 11.4 --> 750x1334 (True resolution)
driver.get_window_size() --> 375 * 667 (scaled resolution)

DETAILS

1. Using find_element_by_image() returns an ImageElement based off of the true resolution of the device. This includes the center_x and center_y values. For example, lets say the values are x=100, y=200

2. Creating a reference image using save_screenshot(), I confirmed the coords from ImageElement are correct. Ie. x=100 and y=200 is the center of the image element.

NOTE: save_screenshot() creates a PNG image with the true resolution

3. When ImageElement.click() is called, I noticed a different element is tapped.
   Doing some debugging, I realized it was tapping on x=100, y=200 on the scaled resolution(375 * 667)

[mykola-mokhnach]

Check if any of image lookup settings are helpful there: https://github.com/appium/appium-base-driver/blob/f510db0257110a755c9f2976bf7580abc9402bc9/lib/basedriver/device-settings.js

[qa_adrian]

thanks for the reply mykola-mokhnach!

I looked into the settings but unfortunately, same results. My settings are:

{
‘imageMatchThreshold’: 0.4,
‘fixImageFindScreenshotDims’: True,
‘fixImageTemplateSize’: False,
‘checkForImageElementStaleness’: True,
‘autoUpdateImageElementPosition’: False,
‘imageElementTapStrategy’: ‘w3cActions’,
}

Toggling the settings for fixImageFindScreenshotDims and fixImageTemplateSize doesn’t seem to produce different results. The ImageElement is always created with:
x 67
y 248
width 666
height 152
center_x 400
center_y 310
The coords above suggest they’re taken from the True Resolution as get_window_size() returns {‘width’: 375, ‘height’: 667}

Setting checkForImageElementStaleness and autoUpdateImageElementPosition to True seems to yield the same results.

[mykola-mokhnach]

@jlipps Can you please take a look?

[Aleksei]

@qa_adrian as workaround you can get center of image element and click yourself by x and y with scaling whatever needed.

[qa_adrian]

@Aleksei sounds good to me. cheers.

Thanks for looking into it everyone