Consecutive touchpad clicks are perceived as one

[sedyh]

Ebitengine Version

2.8+ (6452cbc)

Operating System

• Windows
• macOS
• Linux
• FreeBSD
• OpenBSD
• Android
• iOS
• Nintendo Switch
• PlayStation 5
• Xbox
• Web Browsers

Go Version (go version)

go version go1.23.1 linux/amd64

What steps will reproduce the problem?

Run the example:

package main

import (
	"fmt"
	"log"
	"github.com/hajimehoshi/ebiten/v2"
	"github.com/hajimehoshi/ebiten/v2/ebitenutil"
	"github.com/hajimehoshi/ebiten/v2/inpututil"
)

type Game struct{}

func NewGame() *Game {
	return &Game{}
}

func (g *Game) Update() error {
	just := inpututil.IsMouseButtonJustPressed(ebiten.MouseButtonLeft)
	pressed := ebiten.IsMouseButtonPressed(ebiten.MouseButtonLeft)
	if just {
		log.Printf("just: %t, press: %t\n", just, pressed)
	}
	return nil
}

func (g *Game) Draw(screen *ebiten.Image) {
	ebitenutil.DebugPrint(screen, fmt.Sprintf("FPS: %0.2f", ebiten.ActualFPS()))
}

func (g *Game) Layout(w, h int) (int, int) {
	return w, h
}

func main() {
	log.SetFlags(log.Ltime)
	ebiten.SetWindowSize(480, 320)
	ebiten.SetWindowResizingMode(ebiten.WindowResizingModeEnabled)
	if err := ebiten.RunGame(NewGame()); err != nil {
		panic(err)
	}
}

Run the input monitor with your device id (xinput without flags will list device ids):

#!/usr/bin/bash

xinput test-xi2 --root 12 | while read -r line; do
    timestamp=$(date +%s%3N)
    if [[ "$line" == *"RawButtonPress"* ]]; then
        echo "[$timestamp] $line"
    fi
    if [[ "$line" == *"RawButtonRelease"* ]]; then
        echo "[$timestamp] $line"
    fi
done

What is the expected result?

Every times xinput fires, ebitengine example should fire a log too.

What happens instead?

The engine perceives frequent clicks (>2 cps) as single press:

ebitengine-real-fail.mp4

Anything else you feel useful to add?

No response

[hajimehoshi]

Let me set the milestone v2.9 first, but if the fix is obvious and the issue is serious enough, I might change the mileston to the stable version.

[hajimehoshi]

@sedyh I would like to know which Ebitengine or GLFW was problematic. You checked

ebiten/internal/ui/input_glfw.go

Lines 77 to 83 in 6452cbc

	for gb, ub := range glfwMouseButtonToMouseButton {
	s, err := u.window.GetMouseButton(gb)
	if err != nil {
	return err
	}
	u.inputState.MouseButtonPressed[ub] = s == glfw.Press
	}

is called every frame (144Hz?) and s is true for longer time than you actually pressed, right?

[sedyh]

You checked that code is called every frame (144Hz?) and s is true for longer time than you actually pressed, right?

Yes. Here is some additional output and the code:

for gb, ub := range glfwMouseButtonToMouseButton {
	s, err := u.window.GetMouseButton(gb)
	if err != nil {
		return err
	}
	u.inputState.MouseButtonPressed[ub] = s == glfw.Press
	if gb == 0 {
		log.Println("press:", s)
	}
}

[hajimehoshi]

Are mouse events fired as expected?

ebiten/internal/glfw/x11_window_linbsd.c

Lines 1468 to 1477 in 6452cbc

	case ButtonPress:
	{
	const int mods = translateState(event->xbutton.state);
	if (event->xbutton.button == Button1)
	_glfwInputMouseClick(window, GLFW_MOUSE_BUTTON_LEFT, GLFW_PRESS, mods);
	else if (event->xbutton.button == Button2)
	_glfwInputMouseClick(window, GLFW_MOUSE_BUTTON_MIDDLE, GLFW_PRESS, mods);
	else if (event->xbutton.button == Button3)
	_glfwInputMouseClick(window, GLFW_MOUSE_BUTTON_RIGHT, GLFW_PRESS, mods);

[sedyh]

Are mouse events fired as expected?

Yes, they are.

const int mods = translateState(event->xbutton.state);

printf("event: %d\n", event->xbutton.button);

if (event->xbutton.button == Button1)
    _glfwInputMouseClick(window, GLFW_MOUSE_BUTTON_LEFT, GLFW_PRESS, mods);
else if (event->xbutton.button == Button2)
    _glfwInputMouseClick(window, GLFW_MOUSE_BUTTON_MIDDLE, GLFW_PRESS, mods);
else if (event->xbutton.button == Button3)
    _glfwInputMouseClick(window, GLFW_MOUSE_BUTTON_RIGHT, GLFW_PRESS, mods);

ebitengine-real-clicks.mp4

[hajimehoshi]

@sedyh Are the events actually for mouse buttons?

[sedyh]

Its all just left click:

11:40:46 just: true, press: true
event-button-left: 1
event-button-left: 1
event-button-left: 1
event-button-left: 1
event-button-left: 1
event-button-left: 1

if (event->xbutton.button == Button1) {
    printf("event-button-left: %d\n", event->xbutton.button);
    _glfwInputMouseClick(window, GLFW_MOUSE_BUTTON_LEFT, GLFW_PRESS, mods);
} else if (event->xbutton.button == Button2) {
    _glfwInputMouseClick(window, GLFW_MOUSE_BUTTON_MIDDLE, GLFW_PRESS, mods);
} else if (event->xbutton.button == Button3) {
    _glfwInputMouseClick(window, GLFW_MOUSE_BUTTON_RIGHT, GLFW_PRESS, mods);
}

[hajimehoshi]

My current guess is that this is due to sticky states but I am not 100% sure

ebiten/internal/glfw/input_unix.c

Lines 93 to 94 in 6452cbc

	if (action == GLFW_RELEASE && window->stickyMouseButtons)
	window->mouseButtons[button] = _GLFW_STICK;

[hajimehoshi]

@sedyh Thanks. Are releasing events also fired?

ebiten/internal/glfw/x11_window_linbsd.c

Lines 1502 to 1535 in 6452cbc

	case ButtonRelease:
	{
	const int mods = translateState(event->xbutton.state);
	if (event->xbutton.button == Button1)
	{
	_glfwInputMouseClick(window,
	GLFW_MOUSE_BUTTON_LEFT,
	GLFW_RELEASE,
	mods);
	}
	else if (event->xbutton.button == Button2)
	{
	_glfwInputMouseClick(window,
	GLFW_MOUSE_BUTTON_MIDDLE,
	GLFW_RELEASE,
	mods);
	}
	else if (event->xbutton.button == Button3)
	{
	_glfwInputMouseClick(window,
	GLFW_MOUSE_BUTTON_RIGHT,
	GLFW_RELEASE,
	mods);
	}
	else if (event->xbutton.button > Button7)
	{
	// Additional buttons after 7 are treated as regular buttons
	// We subtract 4 to fill the gap left by scroll input above
	_glfwInputMouseClick(window,
	event->xbutton.button - Button1 - 4,
	GLFW_RELEASE,
	mods);
	}

[sedyh]

@hajimehoshi Yes, for the left button:

12:42:18 just: true, press: true
event-button-release: 1
event-button-press: 1
event-button-release: 1
event-button-press: 1
event-button-release: 1
event-button-press: 1
event-button-release: 1
event-button-press: 1
event-button-release: 1
event-button-press: 1
event-button-release: 1
event-button-press: 1

[sedyh]

My current guess is that this is due to sticky states but I am not 100% sure

Not sure about this:

ebiten/internal/ui/ui_glfw.go

Lines 1263 to 1272 in 6452cbc

	sticky := glfw.True
	if fpsMode == FPSModeVsyncOffMinimum {
	sticky = glfw.False
	}
	if err := u.window.SetInputMode(glfw.StickyMouseButtonsMode, sticky); err != nil {
	return err
	}
	if err := u.window.SetInputMode(glfw.StickyKeysMode, sticky); err != nil {
	return err
	}

ebiten.SetFPSMode(ebiten.FPSModeVsyncOffMinimum)

12:47:10 just: true, press: true
event-button-release: 1
event-button-press: 1
event-button-release: 1
event-button-press: 1
event-button-release: 1
event-button-press: 1
event-button-release: 1
event-button-press: 1
event-button-release: 1
event-button-press: 1
event-button-release: 1
event-button-press: 1
event-button-release: 1

[hajimehoshi]

My current understanding is

• GLFW (X) handls events correctly
• Ebitengine uses the sticky mode. Just after releasing, the button is still pressed, but after GetMouseButton, the state should be reset.
• u.window.GetMouseButton unexpectedly often returns 'pressed'. Why?

As long as I cannot reproduce the issue, it is almost impossible to know what's going on. @sedyh Would it be possible for you to investigate why this mismatch happens?

[sedyh]

Would it be possible for you to investigate why this mismatch happens?

I'll try, but I'll need some hints on the internals. For example: where should I start or what other places can I check?

This fragment looks sus, but it being used on a lower level and only for the keyboard so I guess its not related:

[hajimehoshi]

Thanks. Keyboards has similar implementation but has different stories. Let's focus on mouse buttons first.

[hajimehoshi]

@sedyh and I have discusse in the Discord server and found the following:

Now Ebitengine just watches input states wihtout handling events. The issue is that a release event could be delayed and it could come just before the next pressing event. Ebitengine could not detect the state change when the two events came at one event polling.

@sedyh confimed that another mouse worked as expected, so I think this is an issue in the X driver for the touchpad.

In order to mitigate this issue, my current idea is to fix inpututil.IsMouseButtonJustReleased to return true in such case by handling mouse-button-release events. inpututil.IsMouseButtonJustPressed would no longer be just a useful wrapper but an essential API which cannot be implemented without accessing an internal detail. inpututil.MouseButtonPressDuration would also be affected. I'm not sure we should do the same thing for keyboard and touch APIs.

Unfortunately, ebiten.IsMouseButtonPressed would keep returning true (pressing) in such case.

We might have to add IsMouseButtonJustReleased to the core ebiten package in addition to inpututil later, as this is essential.