Add Alvin library: input, capture, windows and vision
A cross-platform C++23 library with a platform-neutral public API and runtime-selected backends: - Input: Wayland virtual pointer/keyboard (Hyprland, wlroots), libei via the RemoteDesktop portal (GNOME, KDE), uinput, and Win32 SendInput - Capture: ScreenCast portal + PipeWire, and Win32 GDI - Windows: Hyprland IPC and Win32 enumeration - Monitors via wl_output/xdg-output and EnumDisplayMonitors - Optional OpenCV template matching (alvin::vision) Also adds the `alvin` CLI, Catch2 unit tests, install rules with find_package(Alvin) support, API docs (docs/API.md and the Obsidian vault), and README instructions for both ways of consuming the library. Windows backends are written but not yet compiled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Aeth3randClaude Opus 5.5 committed
1 parent
76ff506fd0
commit
ca7e62ac8d
59 files changed
+6515
-1
No files matched your search
@@ -0,0 +1,498 @@
|
||||
# Alvin API Reference
|
||||
|
||||
Everything public lives in namespace `alvin` (vision helpers in `alvin::vision`).
|
||||
Include the umbrella header for the core library:
|
||||
|
||||
```cpp
|
||||
#include <alvin/alvin.hpp> // Input, Capture, WindowManager, Frame, keys, types
|
||||
#include <alvin/vision.hpp> // optional, needs the alvin::vision target (OpenCV)
|
||||
```
|
||||
|
||||
- [[#Using Alvin in your project]]
|
||||
- [[#Error handling]]
|
||||
- [[#Coordinates and monitors]]
|
||||
- [[#Input]]
|
||||
- [[#Keys and chords]]
|
||||
- [[#Capture]]
|
||||
- [[#Frame]]
|
||||
- [[#Windows]]
|
||||
- [[#Vision (OpenCV)]]
|
||||
- [[#Putting it together]]
|
||||
|
||||
---
|
||||
|
||||
## Using Alvin in your project
|
||||
|
||||
Either build Alvin as part of your project, or install it once and use `find_package`.
|
||||
The README's "Using Alvin in your project" section has step-by-step instructions for both.
|
||||
|
||||
```cmake
|
||||
add_subdirectory(external/Alvin) # option 1: build from source
|
||||
# find_package(Alvin 0.1 REQUIRED) # option 2: use an installed Alvin
|
||||
|
||||
target_link_libraries(my_agent PRIVATE alvin::alvin) # core
|
||||
target_link_libraries(my_agent PRIVATE alvin::vision) # + OpenCV helpers, if built
|
||||
```
|
||||
|
||||
Alvin requires C++23 (it uses `std::expected`).
|
||||
|
||||
---
|
||||
|
||||
## Error handling
|
||||
|
||||
Alvin never throws. Every call that can fail returns a `Result<T>`, which is
|
||||
`std::expected<T, alvin::Error>`. Calls with no value return `Status`
|
||||
(`Result<void>`).
|
||||
|
||||
```cpp
|
||||
struct Error {
|
||||
ErrorCode code;
|
||||
std::string message; // human-readable detail
|
||||
};
|
||||
|
||||
enum class ErrorCode {
|
||||
Unknown, Unsupported, NotAvailable, PermissionDenied, Cancelled,
|
||||
Timeout, InvalidArgument, NotFound, Io, Protocol,
|
||||
};
|
||||
|
||||
std::string_view toString(ErrorCode code);
|
||||
```
|
||||
|
||||
| Code | Typical cause |
|
||||
|------|---------------|
|
||||
| `Unsupported` | Backend not compiled in, or the feature isn't possible on this platform |
|
||||
| `NotAvailable` | Backend compiled in but its service is missing (no Wayland display, no portal interface) |
|
||||
| `PermissionDenied` | `/dev/uinput` not writable, portal request denied, Windows UIPI block |
|
||||
| `Cancelled` | The user dismissed a portal dialog |
|
||||
| `Timeout` | No answer from a portal dialog, or no frame arrived in time |
|
||||
| `NotFound` | A window or monitor that doesn't exist |
|
||||
|
||||
Usage patterns:
|
||||
|
||||
```cpp
|
||||
auto input = alvin::Input::create();
|
||||
if (!input) {
|
||||
std::cerr << alvin::toString(input.error().code) << ": " << input.error().message << '\n';
|
||||
return 1;
|
||||
}
|
||||
(*input)->moveTo({100, 100}); // Status; check it or ignore it
|
||||
|
||||
if (auto s = (*input)->click(); !s) // check a Status
|
||||
std::cerr << s.error().message << '\n';
|
||||
|
||||
auto in = alvin::Input::create().value(); // or throw std::bad_expected_access on failure
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Coordinates and monitors
|
||||
|
||||
`#include <alvin/types.hpp>`
|
||||
|
||||
All positions are **global desktop coordinates**:
|
||||
|
||||
- **Linux:** logical pixels, after monitor scaling. These are the numbers `hyprctl` reports.
|
||||
- **Windows:** physical pixels. Alvin makes the process per-monitor DPI aware.
|
||||
|
||||
```cpp
|
||||
struct Point { int x, y; };
|
||||
struct Size { int width, height; };
|
||||
|
||||
struct Rect {
|
||||
int x, y, width, height;
|
||||
int right() const; // x + width
|
||||
int bottom() const; // y + height
|
||||
Point center() const;
|
||||
bool empty() const;
|
||||
bool contains(Point p) const;
|
||||
Rect united(const Rect& o) const; // bounding box of both
|
||||
};
|
||||
|
||||
struct Monitor {
|
||||
std::string name; // "DP-2", "\\.\DISPLAY1"
|
||||
std::string description; // make/model
|
||||
Rect bounds; // position and size on the desktop
|
||||
Size pixelSize; // actual mode, e.g. 3840x2160
|
||||
double scale; // pixelSize / logical size, e.g. 1.333
|
||||
bool primary;
|
||||
};
|
||||
|
||||
Result<std::vector<Monitor>> listMonitors();
|
||||
Result<Rect> desktopBounds(); // bounding box of all monitors
|
||||
```
|
||||
|
||||
```cpp
|
||||
for (const auto& m : alvin::listMonitors().value())
|
||||
std::printf("%s at %d,%d (%dx%d logical, scale %.2f)\n",
|
||||
m.name.c_str(), m.bounds.x, m.bounds.y, m.bounds.width, m.bounds.height, m.scale);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Input
|
||||
|
||||
`#include <alvin/input.hpp>`
|
||||
|
||||
### Creating
|
||||
|
||||
```cpp
|
||||
static Result<std::unique_ptr<Input>> Input::create(const InputOptions& options = {});
|
||||
|
||||
struct InputOptions {
|
||||
InputBackend backend = InputBackend::Auto;
|
||||
std::string restoreToken; // libei only: skip the portal dialog (see restoreToken())
|
||||
std::chrono::milliseconds permissionTimeout = std::chrono::minutes(2);
|
||||
};
|
||||
|
||||
enum class InputBackend { Auto, WlrVirtual, LibEi, UInput, SendInput };
|
||||
```
|
||||
|
||||
| Backend | Platform | Notes |
|
||||
|---------|----------|-------|
|
||||
| `WlrVirtual` | Hyprland, sway, wlroots | No dialog. Types any Unicode text. |
|
||||
| `LibEi` | GNOME, KDE (RemoteDesktop portal) | Shows a permission dialog unless given a restore token. |
|
||||
| `UInput` | Any Linux session | Needs write access to `/dev/uinput`. Takes about 2.5 s to create on Hyprland. |
|
||||
| `SendInput` | Windows | Can't inject into elevated windows unless Alvin runs elevated too. |
|
||||
|
||||
With `Auto`, Linux tries `WlrVirtual`, then `LibEi`, then `UInput`. You can force a
|
||||
backend with the environment variable `ALVIN_INPUT_BACKEND=wlr|libei|uinput` (Linux only).
|
||||
|
||||
Create one `Input` and keep it for the life of your program. Creation is the
|
||||
expensive part (connecting to the compositor, creating devices, portal dialogs).
|
||||
|
||||
### Mouse
|
||||
|
||||
| Call | Description |
|
||||
|------|-------------|
|
||||
| `Status moveTo(Point p)` | Move the pointer to an absolute desktop position. |
|
||||
| `Status moveBy(int dx, int dy)` | Relative motion. With `uinput`, the compositor applies pointer acceleration. |
|
||||
| `Status buttonDown(MouseButton b)` / `buttonUp(b)` | Press or release a button. |
|
||||
| `Status click(MouseButton b = Left, int count = 1)` | Click in place. `count = 2` double-clicks. |
|
||||
| `Status clickAt(Point p, MouseButton b = Left, int count = 1)` | Move, then click. |
|
||||
| `Status drag(Point from, Point to, MouseButton b = Left)` | Press at `from`, move to `to` in 10 steps, release. |
|
||||
| `Status scroll(int dx, int dy)` | Wheel notches. Positive `dy` scrolls **down**, positive `dx` scrolls **right**. |
|
||||
|
||||
`MouseButton` is `Left`, `Right`, `Middle`, `Back` or `Forward`.
|
||||
|
||||
```cpp
|
||||
input->clickAt({1440, 810}); // left click
|
||||
input->clickAt({1440, 810}, alvin::MouseButton::Right);
|
||||
input->click(alvin::MouseButton::Left, 2); // double-click in place
|
||||
input->drag({100, 100}, {600, 400}); // e.g. select or move something
|
||||
input->scroll(0, 3); // 3 notches down
|
||||
```
|
||||
|
||||
### Keyboard
|
||||
|
||||
| Call | Description |
|
||||
|------|-------------|
|
||||
| `Status keyDown(Key k)` / `keyUp(Key k)` | Press or release one physical key. |
|
||||
| `Status keyPress(Key k)` | Down, then up. |
|
||||
| `Status chord(std::initializer_list<Key>)` | Press in order, release in reverse. |
|
||||
| `Status chord(std::string_view)` | Same, from text like `"ctrl+shift+t"`. |
|
||||
| `Status typeText(std::string_view utf8)` | Type text. `\n` presses Enter and `\t` presses Tab. |
|
||||
|
||||
```cpp
|
||||
using alvin::Key;
|
||||
input->keyPress(Key::Enter);
|
||||
input->chord({Key::LeftCtrl, Key::C});
|
||||
input->chord("ctrl+shift+t");
|
||||
input->chord("super+enter");
|
||||
input->typeText("Hello, wörld ✓\n");
|
||||
```
|
||||
|
||||
`chord()` always releases every key it pressed, even when an earlier call fails,
|
||||
so a key is never left held down.
|
||||
|
||||
**How `typeText` handles layouts:**
|
||||
|
||||
- **`wlr`:** builds a keymap containing exactly the characters needed, so any Unicode text works.
|
||||
- **`libei`:** uses the server's keyboard layout. If the server supports libei's text capability, it sends the text directly.
|
||||
- **`uinput`:** uses the layout named in `XKB_DEFAULT_LAYOUT`/`XKB_DEFAULT_VARIANT` (default `us`). Characters that aren't on that layout fail with `Unsupported` before anything is typed.
|
||||
- **`sendinput`:** sends Unicode directly, so any text works.
|
||||
|
||||
### Other
|
||||
|
||||
| Call | Description |
|
||||
|------|-------------|
|
||||
| `InputBackend backend() const` | Which backend was chosen. |
|
||||
| `std::string restoreToken() const` | libei: save this and pass it in `InputOptions::restoreToken` next run to skip the dialog. Empty for other backends. |
|
||||
| `void setEventDelay(std::chrono::milliseconds)` | Pause between the down and up halves of clicks, key presses and typed characters. The default is 8 ms. Raise it for applications that miss fast input. |
|
||||
| `std::string_view toString(InputBackend)` | `"wlr"`, `"libei"`, `"uinput"`, `"sendinput"`, `"auto"` |
|
||||
|
||||
---
|
||||
|
||||
## Keys and chords
|
||||
|
||||
`#include <alvin/keys.hpp>`
|
||||
|
||||
`Key` names **physical key positions** on a US QWERTY layout, the same way evdev
|
||||
and Win32 scan codes do. Use `typeText()` when you want particular characters.
|
||||
|
||||
```
|
||||
A–Z, Num0–Num9, F1–F24,
|
||||
Escape, Enter, Tab, Backspace, Space,
|
||||
Minus, Equal, LeftBracket, RightBracket, Backslash, Semicolon, Apostrophe, Grave, Comma, Period, Slash,
|
||||
CapsLock, NumLock, ScrollLock,
|
||||
LeftShift, RightShift, LeftCtrl, RightCtrl, LeftAlt, RightAlt, LeftSuper, RightSuper, Menu,
|
||||
Insert, Delete, Home, End, PageUp, PageDown, Left, Right, Up, Down, PrintScreen, Pause,
|
||||
Kp0–Kp9, KpDecimal, KpDivide, KpMultiply, KpSubtract, KpAdd, KpEnter,
|
||||
VolumeMute, VolumeDown, VolumeUp, MediaPlayPause, MediaStop, MediaNext, MediaPrevious
|
||||
```
|
||||
|
||||
```cpp
|
||||
std::optional<Key> keyFromName(std::string_view name); // case-insensitive
|
||||
std::string_view keyName(Key key);
|
||||
std::optional<std::vector<Key>> parseChord(std::string_view chord);
|
||||
bool isModifier(Key key);
|
||||
```
|
||||
|
||||
Names accepted by `keyFromName` and in chord strings:
|
||||
|
||||
| Key | Names |
|
||||
|-----|-------|
|
||||
| Letters and digits | `a`…`z`, `0`…`9` |
|
||||
| Function keys | `f1`…`f24` |
|
||||
| Modifiers | `shift`/`lshift`, `rshift`, `ctrl`/`control`/`lctrl`, `rctrl`, `alt`/`lalt`, `ralt`/`altgr`, `super`/`win`/`meta`/`cmd`/`lsuper`, `rsuper` |
|
||||
| Editing | `enter`/`return`, `esc`/`escape`, `tab`, `backspace`, `space`, `insert`, `delete`/`del` |
|
||||
| Navigation | `home`, `end`, `pageup`/`pgup`, `pagedown`/`pgdn`, `left`, `right`, `up`, `down` |
|
||||
| Punctuation | `minus`/`-`, `equal`/`=`, `[`, `]`, `\`, `;`, `'`, `` ` ``, `,`, `.`, `/` (or their word names) |
|
||||
| Keypad | `kp0`…`kp9`, `kpdecimal`, `kpdivide`, `kpmultiply`, `kpsubtract`, `kpadd`, `kpenter` |
|
||||
| Media | `mute`, `volumeup`, `volumedown`, `playpause`, `stop`, `next`, `prev`/`previous` |
|
||||
| Other | `capslock`, `numlock`, `scrolllock`, `printscreen`/`print`, `pause`, `menu` |
|
||||
|
||||
Chords are joined with `+`, and spaces around keys are ignored. `"ctrl++"` means
|
||||
Ctrl plus the `+`/`=` key.
|
||||
|
||||
---
|
||||
|
||||
## Capture
|
||||
|
||||
`#include <alvin/capture.hpp>`
|
||||
|
||||
### Creating
|
||||
|
||||
```cpp
|
||||
static Result<std::unique_ptr<Capture>> Capture::create(const CaptureOptions& options = {});
|
||||
|
||||
struct CaptureOptions {
|
||||
CaptureBackend backend = CaptureBackend::Auto; // Auto, Portal, Gdi
|
||||
CaptureSource source = CaptureSource::Monitor; // Monitor, Window, Any
|
||||
int monitorIndex = 0; // Windows: which monitor (index into listMonitors())
|
||||
std::optional<WindowId> window; // Windows: capture this window directly
|
||||
bool showCursor = false;
|
||||
std::string restoreToken; // Portal: skip the picker (see below)
|
||||
std::chrono::milliseconds permissionTimeout = std::chrono::minutes(2);
|
||||
};
|
||||
```
|
||||
|
||||
**Linux (`Portal`):** `create()` opens the desktop's share picker. On Hyprland
|
||||
that's `hyprland-share-picker`. `source` controls whether the picker offers
|
||||
monitors, windows or both. When the call returns, a PipeWire stream is running
|
||||
in the background.
|
||||
|
||||
**Windows (`Gdi`):** there's no dialog. Set `monitorIndex`, or set `window` to
|
||||
capture one window.
|
||||
|
||||
### Skipping the picker next time
|
||||
|
||||
```cpp
|
||||
alvin::CaptureOptions opts;
|
||||
opts.restoreToken = loadSavedToken(); // empty the first time
|
||||
auto cap = alvin::Capture::create(opts).value();
|
||||
saveToken(cap->restoreToken()); // new token each session; always re-save it
|
||||
```
|
||||
|
||||
The `alvin` CLI does this for you and stores tokens in `~/.cache/alvin/`.
|
||||
|
||||
> Don't leave a picker unanswered. While it's open, xdg-desktop-portal-hyprland
|
||||
> blocks every other portal request on the system, screen sharing in other
|
||||
> apps included. Use `permissionTimeout` to limit how long `create()` waits.
|
||||
|
||||
### Getting frames
|
||||
|
||||
| Call | Description |
|
||||
|------|-------------|
|
||||
| `Result<Frame> grab(timeout = 5s)` | The most recent frame. Blocks only until the first frame exists. |
|
||||
| `Result<Frame> nextFrame(timeout = 5s)` | Waits for a frame newer than the last one you received. |
|
||||
| `std::string restoreToken() const` | Portal token for next time. |
|
||||
| `CaptureBackend backend() const` | Which backend was chosen. |
|
||||
|
||||
PipeWire streams only send a frame when the screen changes, so `grab()` can
|
||||
return an image that's a few seconds old if nothing has moved. That's usually
|
||||
what you want. After you act (click, type), call `nextFrame()` to see the result:
|
||||
|
||||
```cpp
|
||||
input->clickAt(button);
|
||||
auto after = cap->nextFrame(std::chrono::seconds(2));
|
||||
if (!after && after.error().code == alvin::ErrorCode::Timeout) {
|
||||
// nothing on screen changed
|
||||
}
|
||||
```
|
||||
|
||||
A `Capture` holds a live stream, so keep one and reuse it. Don't create one per frame.
|
||||
|
||||
---
|
||||
|
||||
## Frame
|
||||
|
||||
`#include <alvin/frame.hpp>`
|
||||
|
||||
A `Frame` is an owned BGRA8 image (4 bytes per pixel: B, G, R, A, with no row
|
||||
padding). It also records where it sits on the desktop.
|
||||
|
||||
| Member | Description |
|
||||
|--------|-------------|
|
||||
| `Frame(int w, int h)` | New zero-filled frame. |
|
||||
| `int width()`, `int height()`, `Size size()` | Size in **pixels**. |
|
||||
| `std::size_t stride()` | Bytes per row (`width * 4`). |
|
||||
| `uint8_t* data()`, `row(y)`, `std::span<const uint8_t> bytes()` | Raw pixel access. |
|
||||
| `Color pixel(int x, int y)` | `{r, g, b, a}`. Returns all zeros if out of bounds. |
|
||||
| `Rect desktopRect()` | The area of the desktop this frame shows, in desktop coordinates. |
|
||||
| `Point toDesktop(Point px)` | Frame pixel to desktop coordinate. |
|
||||
| `Point fromDesktop(Point p)` | Desktop coordinate to frame pixel. |
|
||||
| `Frame crop(Rect r)` | Sub-image (`r` is in frame pixels and gets clipped). `desktopRect()` is adjusted to match. |
|
||||
| `int64_t timestampNs()` | Capture timestamp. |
|
||||
| `Status savePpm(path)` | Save without dependencies (PPM). With vision, use `vision::saveImage` for PNG/JPG. |
|
||||
|
||||
**Always convert through `toDesktop()` before clicking.** On a monitor scaled
|
||||
to 1.33, a 3840×2160 frame covers 2880×1620 desktop units, so frame pixels
|
||||
aren't desktop coordinates:
|
||||
|
||||
```cpp
|
||||
auto frame = cap->grab().value();
|
||||
alvin::Point pixel{1920, 1080}; // something found in the image
|
||||
input->clickAt(frame.toDesktop(pixel)); // correct on any scale
|
||||
```
|
||||
|
||||
```cpp
|
||||
alvin::Color c = frame.pixel(10, 10);
|
||||
if (c.r > 200 && c.g < 50 && c.b < 50) { /* red */ }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Windows
|
||||
|
||||
`#include <alvin/window.hpp>`
|
||||
|
||||
Wayland hides other windows from normal clients, so each compositor needs its
|
||||
own backend. Right now that's Hyprland (IPC socket) and Win32.
|
||||
|
||||
```cpp
|
||||
static Result<std::unique_ptr<WindowManager>> WindowManager::create(WindowBackend b = WindowBackend::Auto);
|
||||
|
||||
struct WindowInfo {
|
||||
WindowId id; // opaque handle; pass it to focus()
|
||||
std::string title;
|
||||
std::string appId; // Hyprland class / Win32 executable name
|
||||
Rect bounds; // desktop coordinates
|
||||
int64_t pid;
|
||||
bool focused, visible, minimized, fullscreen;
|
||||
std::string workspace; // Hyprland workspace name
|
||||
};
|
||||
```
|
||||
|
||||
| Call | Description |
|
||||
|------|-------------|
|
||||
| `Result<std::vector<WindowInfo>> list()` | All mapped top-level windows. |
|
||||
| `Result<std::optional<WindowInfo>> active()` | The focused window, or none. |
|
||||
| `Result<std::optional<WindowInfo>> find(std::string_view text)` | First window whose title or appId contains `text` (case-insensitive). |
|
||||
| `Status focus(WindowId id)` | Focus it. On Hyprland this switches to its workspace. |
|
||||
|
||||
```cpp
|
||||
auto wm = alvin::WindowManager::create().value();
|
||||
if (auto w = wm->find("firefox").value()) {
|
||||
wm->focus(w->id);
|
||||
input->clickAt(w->bounds.center());
|
||||
// click 40px in from the top-left of the window
|
||||
input->clickAt({w->bounds.x + 40, w->bounds.y + 40});
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Vision (OpenCV)
|
||||
|
||||
`#include <alvin/vision.hpp>`, link `alvin::vision`.
|
||||
|
||||
| Call | Description |
|
||||
|------|-------------|
|
||||
| `cv::Mat view(const Frame&)` | Zero-copy `CV_8UC4` view. The Frame must outlive the Mat. |
|
||||
| `cv::Mat toBgr(const Frame&)` | Deep copy as `CV_8UC3`, the layout most OpenCV code expects. |
|
||||
| `Result<Frame> fromMat(const cv::Mat&)` | 8-bit gray, BGR or BGRA to a Frame. |
|
||||
| `Result<cv::Mat> loadImage(path)` | Load a PNG/JPG as BGR. |
|
||||
| `Status saveImage(const Frame&, path)` | Save as PNG/JPG/… (by extension). |
|
||||
| `Result<std::optional<Match>> find(frame, needle, opts)` | Best template match at or above the threshold. |
|
||||
| `Result<std::vector<Match>> findAll(frame, needle, opts)` | Every non-overlapping match, best first. |
|
||||
|
||||
```cpp
|
||||
struct Match { Rect rect; double score; }; // rect in FRAME PIXELS, score 0..1
|
||||
|
||||
struct MatchOptions {
|
||||
double threshold = 0.9; // lower it (0.7–0.85) if the target is anti-aliased or slightly different
|
||||
bool grayscale = false; // faster; ignores colour
|
||||
int maxResults = 16; // findAll only
|
||||
};
|
||||
```
|
||||
|
||||
Take template images from Alvin's own captures, e.g. `alvin capture` followed by
|
||||
a crop. That keeps the scale and colours identical, which template matching needs.
|
||||
|
||||
```cpp
|
||||
auto needle = alvin::vision::loadImage("save_button.png").value();
|
||||
auto frame = cap->grab().value();
|
||||
if (auto hit = alvin::vision::find(frame, needle, {.threshold = 0.85}).value())
|
||||
input->clickAt(frame.toDesktop(hit->rect.center()));
|
||||
|
||||
for (const auto& m : alvin::vision::findAll(frame, needle, {.threshold = 0.8}).value())
|
||||
std::printf("match at %d,%d score %.2f\n", m.rect.x, m.rect.y, m.score);
|
||||
```
|
||||
|
||||
For anything else OpenCV can do, use `vision::view(frame)` to get a `cv::Mat`.
|
||||
|
||||
---
|
||||
|
||||
## Putting it together
|
||||
|
||||
A small agent loop: wait for a button to appear, click it, then check that the
|
||||
screen changed.
|
||||
|
||||
```cpp
|
||||
#include <alvin/alvin.hpp>
|
||||
#include <alvin/vision.hpp>
|
||||
#include <chrono>
|
||||
#include <iostream>
|
||||
#include <thread>
|
||||
|
||||
using namespace std::chrono_literals;
|
||||
|
||||
int main() {
|
||||
auto input = alvin::Input::create();
|
||||
if (!input) { std::cerr << input.error().message << '\n'; return 1; }
|
||||
|
||||
auto cap = alvin::Capture::create(); // the picker appears on first run
|
||||
if (!cap) { std::cerr << cap.error().message << '\n'; return 1; }
|
||||
|
||||
auto button = alvin::vision::loadImage("accept.png");
|
||||
if (!button) { std::cerr << button.error().message << '\n'; return 1; }
|
||||
|
||||
for (int attempt = 0; attempt < 30; ++attempt) {
|
||||
auto frame = (*cap)->grab();
|
||||
if (!frame) { std::cerr << frame.error().message << '\n'; return 1; }
|
||||
|
||||
auto hit = alvin::vision::find(*frame, *button, {.threshold = 0.85});
|
||||
if (hit && *hit) {
|
||||
(void)(*input)->clickAt(frame->toDesktop((*hit)->rect.center()));
|
||||
if ((*cap)->nextFrame(2s)) std::cout << "clicked; screen updated\n";
|
||||
return 0;
|
||||
}
|
||||
std::this_thread::sleep_for(1s);
|
||||
}
|
||||
std::cerr << "button never appeared\n";
|
||||
return 1;
|
||||
}
|
||||
```
|
||||
|
||||
See `examples/alvin_cli.cpp` in the repo for a complete program
|
||||
that uses every part of the API.
|
||||
Reference in new issue
Block a user