# 2.4 Control Methods

This document provides detailed explanations of the various Screencap (screenshot) and Input (control) methods in MaaFramework and their configurations.

> [!TIP]
>
> - For APIs, screencap/input uses `int` type (bitwise OR combination); for [ProjectInterface V2](3.3-ProjectInterfaceV2.md), use `string` type (use the name directly).
> - [ProjectInterface V2](3.3-ProjectInterfaceV2.md) only supports configuring screencap/mouse/keyboard methods for Win32 controllers. For Adb controllers, screencap/input uses `MaaToolkitAdbDeviceFind` to automatically detect and select the optimal method, no manual configuration required.

## Adb

### Adb Input

> Reference: [MaaDef.h](https://github.com/MaaXYZ/MaaFramework/blob/main/include/MaaFramework/MaaDef.h#L267-L294)

Combine the selected methods below using **bitwise OR** to provide a single value. MaaFramework will try all provided methods in a fixed priority order and select the first available method.

By default, all methods except EmulatorExtras are attempted.

> Priority: EmulatorExtras > Maatouch > MinitouchAndAdbKey > AdbShell

| Name | API Value | Speed | Compatibility | Description |
| --- | --- | --- | --- | --- |
| AdbShell | `1` | Slow | High | |
| MinitouchAndAdbKey | `2` | Fast | Medium | Key input still uses AdbShell |
| Maatouch | `4` | Fast | Medium | |
| EmulatorExtras | `8` | Fast | Low | Only supports emulators: MuMu 12, Tencent App Store (应用宝) |

### Adb Screencap

> Reference: [MaaDef.h](https://github.com/MaaXYZ/MaaFramework/blob/main/include/MaaFramework/MaaDef.h#L230-L266)

Combine the selected methods below using **bitwise OR** to provide a single value. MaaFramework will try all provided methods and select the fastest available method.

By default, all methods except `RawByNetcat`, `MinicapDirect`, and `MinicapStream` are attempted.

`MinicapDirect` and `MinicapStream` encode to jpg (lossy compression), which significantly reduces template matching effectiveness and are not recommended.

| Name | API Value | Speed | Compatibility | Encoding | Description |
| --- | --- | --- | --- | --- | --- |
| EncodeToFileAndPull | `1` | Slow | High | Lossless | |
| Encode | `2` | Slow | High | Lossless | |
| RawWithGzip | `4` | Medium | High | Lossless | |
| RawByNetcat | `8` | Fast | Low | Lossless | |
| MinicapDirect | `16` | Fast | Low | Lossy | |
| MinicapStream | `32` | Very Fast | Low | Lossy | |
| EmulatorExtras | `64` | Very Fast | Low | Lossless | Only supports emulators: MuMu 12, LDPlayer 9, AVD, and Tencent App Store (应用宝) |

## Android Native

The Android native controller uses `MaaAndroidNativeControlUnit` for direct screenshot and input on Android.

### Android Native Prerequisites

1. Android platform only

### Android Native Configuration

Pass a JSON config to `MaaAndroidNativeControllerCreate(config_json)`:

- `library_path`: Android native control unit library path
- `screen_resolution.width` / `screen_resolution.height`: raw screenshot resolution, also used as the touch coordinate space
- `display_id`: target display id, optional, defaults to `0`
- `force_stop`: whether to force stop before `start_app`, optional, defaults to `false`

> [!NOTE]
>
> MaaFramework converts scaled recognition coordinates back to raw screenshot coordinates in `ControllerAgent`.
> The Android native controller no longer performs any frame-to-touch remapping internally, and only clamps to `screen_resolution`.
> If the control unit reports a raw frame resolution different from `screen_resolution`, screencap fails immediately.

## Win32

### Win32 Input

> Reference: [MaaDef.h](https://github.com/MaaXYZ/MaaFramework/blob/main/include/MaaFramework/MaaDef.h#L326-L374)

Select one of the values below.

No default value. Client can choose one as the default.

Different programs on Win32 handle input differently, so there is no universal method.

| Name | API Value | Compatibility | Requires Admin | Seizes Cursor | Background | Description |
| --- | --- | --- | --- | --- | --- | --- |
| Seize | `1` | High | No | Yes | No | |
| SendMessage | `2` | Medium | Maybe | No | Yes | |
| PostMessage | `4` | Medium | Maybe | No | Yes | |
| LegacyEvent | `8` | Low | No | Yes | No | |
| PostThreadMessage | `16` | Low | Maybe | No | Yes | Deprecated |
| SendMessageWithCursorPos | `32` | Medium | Maybe | Brief | Yes | Briefly moves cursor to target position then restores, for apps that check real cursor position |
| PostMessageWithCursorPos | `64` | Medium | Maybe | Brief | Yes | Briefly moves cursor to target position then restores, for apps that check real cursor position |
| SendMessageWithWindowPos | `128` | Medium | Maybe | No | Yes | Briefly moves window to align target with cursor then restores, does not seize mouse |
| PostMessageWithWindowPos | `256` | Medium | Maybe | No | Yes | Briefly moves window to align target with cursor then restores, does not seize mouse |

> [!NOTE]
>
> - Admin privileges mainly depend on the target program's permission level. If the target program runs as administrator, running with admin privileges is required for compatibility.
> - The `WithCursorPos` methods briefly move the cursor to the target position, then restore it after sending the message, hence "Brief" cursor seizure, but it does not block user operations.
> - The `WithWindowPos` methods briefly move the window so the target position aligns with the current cursor position, then restore the window position after sending the message. The cursor is not moved, so there is no mouse seizure, but the window may briefly flicker.
> - Win32 also provides a **Mouse Lock Follow** mode: enable with `MaaControllerSetOption(ctrl, MaaCtrlOption_MouseLockFollow, &enabled, sizeof(bool))` (set `enabled` to `true` to enable, `false` to disable). Designed for TPS/FPS games that lock the mouse to the window in the background. When enabled, the window continuously follows the mouse cursor, and RawInput counter-moves prevent the game from sensing hardware mouse movement. Use `MaaControllerPostRelativeMove` to inject intentional camera rotation while this mode is active. **Note:** On Win32, `MaaControllerPostRelativeMove` requires mouse-lock-follow mode to be active; calling it without this mode will fail. Only supported with MessageInput-based input methods (SendMessage / PostMessage variants).
> - Win32 also provides a **Background Managed Keys** guardian: declare the managed key domain through `MaaControllerSetOption(ctrl, MaaCtrlOption_BackgroundManagedKeys, keycodes, sizeof(int32_t) * count)` with an array of virtual key codes to manage. After setting, matching key operations automatically route through the background guardian path and continuously correct key state while the controller is idle.

### Win32 Screencap

> Reference: [MaaDef.h](https://github.com/MaaXYZ/MaaFramework/blob/main/include/MaaFramework/MaaDef.h#L295-L325)

Combine the selected methods below using **bitwise OR** to provide a single value. MaaFramework will try all provided methods and select the fastest available method.

No default value. Client can choose one combination as the default.

Different programs on Win32 handle rendering differently, so there is no universal method.

| Name | API Value | Speed | Compatibility | Requires Admin | Background | Description |
| --- | --- | --- | --- | --- | --- | --- |
| GDI | `1` | Fast | Medium | No | No | |
| FramePool | `2` | Very Fast | Medium | No | Yes | Available on Windows 10 1903+ |
| DXGI_DesktopDup | `4` | Very Fast | Low | No | No | Desktop duplication (full-screen output copy) |
| DXGI_DesktopDup_Window | `8` | Very Fast | Low | No | No | Desktop duplication cropped to window |
| PrintWindow | `16` | Medium | Medium | No | Yes | |
| ScreenDC | `32` | Fast | High | No | No | |

> [!NOTE]
>
> Three predefined combination macros are provided:  
>
> - `MaaWin32ScreencapMethod_All`: all screencap methods  
> - `MaaWin32ScreencapMethod_Foreground`: `DXGI_DesktopDup_Window | ScreenDC`  
> - `MaaWin32ScreencapMethod_Background`: `FramePool | PrintWindow`  
>
> FramePool and PrintWindow have built-in pseudo-minimize support: when the target window is minimized, they make it transparent and click-through, then restore it without activation, allowing screencap to continue without disturbing the user.  
> Other screencap methods will fail when the target window is minimized. Please avoid minimizing the window when using those methods.

## MacOS

The MacOS controller is used to control native macOS applications on macOS.

### MacOS Prerequisites

1. macOS 14.0 or later
2. The following permissions need to be granted:
   - **Screen Recording permission**: Required for screenshot functionality
   - **Accessibility permission**: Required for input control functionality

### Permission Debugging

If you encounter permission-related issues, you can reset permissions using the following commands:

```bash
# Reset screen recording permission
tccutil reset ScreenCapture

# Reset accessibility permission
tccutil reset Accessibility
```

> [!TIP]
>
> After resetting permissions, you need to restart the application and re-grant the permissions.

> [!TIP]
>
> MaaFramework is not responsible for requesting or guiding permissions; clients must implement these functionalities themselves. MaaToolkit provides a set of helper functions for this purpose; please refer to [test/macos_test](https://github.com/MaaXYZ/MaaFramework/tree/main/test/macos_test).

### MacOS Screencap

> Reference: [MaaDef.h](https://github.com/MaaXYZ/MaaFramework/blob/main/include/MaaFramework/MaaDef.h#L363-L377)

Select one of the values below.

No default value. Client can choose one as the default.

| Name | API Value | Speed | Compatibility | Requires Permission | Background | Description |
| --- | --- | --- | --- | --- | --- | --- |
| ScreenCaptureKit | `1` | Fast | High | Screen Recording | Yes | macOS 14.0+ |

### MacOS Input

> Reference: [MaaDef.h](https://github.com/MaaXYZ/MaaFramework/blob/main/include/MaaFramework/MaaDef.h#L378-L392)

Select one of the values below.

No default value. Client can choose one as the default.

| Name | API Value | Compatibility | Requires Permission | Background | Description |
| --- | --- | --- | --- | --- | --- |
| GlobalEvent | `1` | High | Accessibility | No | Injects into the global HID event stream via `CGEventPost(kCGHIDEventTap)`, dispatched by the OS to the front window (automatically activates the target window) |
| PostToPid | `2` | Medium | Accessibility | Yes | Sends directly to the target process via `CGEventPostToPid`, no need for the target window to be in the foreground |

## PlayCover (macOS)

The PlayCover controller is used to control iOS applications running via [fork PlayCover](https://github.com/hguandl/PlayCover/releases) on macOS.

### PlayCover Prerequisites

1. Install [fork PlayCover](https://github.com/hguandl/PlayCover/releases) on macOS
2. Target iOS app must have MaaTools feature enabled in PlayCover

## Gamepad (Windows)

The Gamepad controller is used to emulate Xbox 360 or DualShock 4 gamepad input on Windows, suitable for games that require gamepad control.

### Gamepad Prerequisites

[ViGEm Bus Driver](https://github.com/ViGEm/ViGEmBus/releases) must be installed.

### Gamepad Types

| Type | API Value | Description |
| --- | --- | --- |
| Xbox360 | `0` | Microsoft Xbox 360 Controller (wired) |
| DualShock4 | `1` | Sony DualShock 4 Controller (wired) |

### Control Mapping

The Gamepad controller uses the following mapping:

#### Digital Buttons (click_key/key_down/key_up)

Use `MaaGamepadButton_*` constants as key values. Xbox button values are used for Xbox360 gamepad, DS4 face buttons automatically map to Xbox equivalents:

| Button | API Value | Xbox 360 | DualShock 4 Equivalent |
| --- | --- | --- | --- |
| DPAD_UP | `1` | D-pad Up | D-pad Up |
| DPAD_DOWN | `2` | D-pad Down | D-pad Down |
| DPAD_LEFT | `4` | D-pad Left | D-pad Left |
| DPAD_RIGHT | `8` | D-pad Right | D-pad Right |
| START / OPTIONS | `16` | Start | Options |
| BACK / SHARE | `32` | Back | Share |
| LEFT_THUMB / L3 | `64` | Left Stick Press | L3 |
| RIGHT_THUMB / R3 | `128` | Right Stick Press | R3 |
| LB / L1 | `256` | LB | L1 |
| RB / R1 | `512` | RB | R1 |
| GUIDE | `1024` | Xbox Guide | - |
| A / CROSS | `4096` | A | ✗ (Cross) |
| B / CIRCLE | `8192` | B | ○ (Circle) |
| X / SQUARE | `16384` | X | □ (Square) |
| Y / TRIANGLE | `32768` | Y | △ (Triangle) |
| PS | `65536` | - | PS (DS4 only) |
| TOUCHPAD | `131072` | - | Touchpad Press (DS4 only) |

#### Analog Input (touch_down/touch_move/touch_up)

Use the `contact` parameter to select the control target:

| Contact | Description | x/y Range | pressure Range |
| --- | --- | --- | --- |
| 0 | Left Stick | -32768 ~ 32767 | Ignored |
| 1 | Right Stick | -32768 ~ 32767 | Ignored |
| 2 | Left Trigger (LT/L2) | Ignored | 0 ~ 255 |
| 3 | Right Trigger (RT/R2) | Ignored | 0 ~ 255 |

> [!NOTE]
>
> - ViGEm Bus Driver must be installed to use this controller.

## WlRoots (Linux)

The WlRoots controller is used to control applications running in a wlroots compositor on Linux.

### Compositor Requirements

The wlroots compositor to be controlled must support the following protocols:

- Wayland Core Protocols
- `virtual-keyboard-unstable-v1`
- `wlr-screencopy-unstable-v1`
- `wlr-virtual-pointer-unstable-v1`

> [!NOTE]
>
> It is recommended to start a nested compositor session to use this controller.
> Controlling the compositor currently used by the desktop is not recommended, as this may lead to unexpected behavior during task execution.

### Keyboard input notice

Keycodes for MaaControllerPostKey{Down,Up} in the WlRoots controller are **by default** evdev key codes, defined in [linux/input-event-codes.h](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/input-event-codes.h).

If you prefer using Win32 Virtual-Key codes (VK_*), pass `true` for the `use_win32_vk_code` parameter when calling `MaaWlRootsControllerCreate`. When enabled, key codes passed to `MaaControllerPostClickKey` / `MaaControllerPostKey{Down,Up}` are interpreted as Win32 VK codes and translated to evdev codes internally. This flag is decided at creation time and cannot be changed at runtime; it makes it easy to reuse key handling logic written for the Win32 controller.
