前言
最近涉及海圖的功能交互,多點觸摸又開始找麻煩。
在PC/Web平臺awtk是通過底層的sdl2庫來實現多點觸摸,但是在嵌入式Linux平臺,可能是考慮到性能原因,awtk并沒有采用sdl庫來做事件處理,而是自己實現一個awtk-linux-fb來做適配,多點觸摸的相關邏輯必須自己適配。
去年11月的時候,匆忙趕工,自己適配了一份tslib的代碼,思路是循環線程內讀取觸點數據后,直接調用awtk自帶的一個multi_gesture.inc文件去計算距離和旋轉角度,在應用層注冊EVT_MULTI_GESTURE事件。
static ret_t tslib_dispatch_one_event(run_info_t* info) {#ifdef MT_TOUCH_ENABLEint ret = -1;if (info->ts != NULL) {ret = ts_read_mt(info->ts, info->samp_mt, info->max_slots, info->read_samples);}uint8_t event_number = 0, down_number = 0;...int max_slots = info->max_slots;// note: down, up only trigger once(ET), move will be keep trigger(LT)for (int i = 0; i < max_slots; i++) {// printf(YELLOW "sample %d - %ld.%06ld -" RESET " (slot %d) %6d %6d %6d\n",// 0,// info->samp_mt[0][i].tv.tv_sec,// info->samp_mt[0][i].tv.tv_usec,// info->samp_mt[0][i].slot,// info->samp_mt[0][i].x,// info->samp_mt[0][i].y,// info->samp_mt[0][i].pressure);if (!(info->samp_mt[0][i].valid & TSLIB_MT_VALID)){//for boards that no clear pressure when slot is invaild, need to do it manuallyif(!s_points[i].type == EVT_MULTI_TOUCH_DOWN){info->samp_mt[0][i].pressure = 0;}continue;}event_number++;}for (int i = 0; i < CT_MAX_TOUCH; i++){s_points[i].touch_id = 0;s_points[i].finger_id = i;if(info->samp_mt[0][i].pressure > 0) {down_number++;s_points[i].type = (s_points[i].x == info->samp_mt[0][i].x && s_points[i].y == info->samp_mt[0][i].y) ? EVT_MULTI_TOUCH_DOWN : EVT_MULTI_TOUCH_MOVE;s_points[i].x = info->samp_mt[0][i].x;s_points[i].y = info->samp_mt[0][i].y;}else{s_points[i].type = EVT_MULTI_TOUCH_UP;s_points[i].x = 0;s_points[i].y = 0;} }if(event_number == 1 && (info->last_down_number == 0 || info->last_down_number == 1)){//單點觸摸...tslib_dispatch(info);}else{//多點觸摸main_loop_t *loop = (main_loop_t*)info->dispatch_ctx;//multi_gesture.inc提供的函數multi_gesture_post_event_from_points(loop, touch_points, down_number, s_points);}
...
后面實測multi_gesture.inc完全不堪用,機器上雙指縮放判定經常失準,雙指拉大,會出現一會放大一會縮小的情況,multi_gesture.inc的業務代碼我沒有細看,但我估計邏輯肯定是有問題的,至少在現在的機器上無法使用。
適配層設計
只能尋找新的觸摸方案了,這時候老板指出來我對觸摸屏的理解有誤,我一開始的想法是手指一觸摸屏幕,立刻會在應用層生成一份即時的完整的觸點數組數據對象,通過這個對象來獲取各觸點坐標,狀態(按下,移動,彈起),按下手指的數量。后面才知道,觸摸屏是逐個去解析按下的各個手指的數據然后處理的,并不是一次就能全部讀取到,也就是說,我想像的這種數據對象的組裝只能到應用層去做而不是適配層去做,之前的思路相當于把解析和業務邏輯都在適配層做,結果業務一復雜就變得不敷使用了。
照著這種思路,應該是適配層只需要把所有手指的數據一個個上報上去,業務層實現實際邏輯,這樣適配層的代碼邏輯會更加簡單,那么AWTK有沒有代表單個手指的數據?
查閱AWTK代碼, 發現有一個touch_event正好符合要求,很幸運也很諷刺,awtk官方在去年12月4日的更新里面已經支持在awtk-linux-fb對touch_event事件上報,還在awtk-web整了一個多點觸摸的演示demo,里面的finger_status_t已經把我設想的數據對象的功能給完成了,我累死累活自己適配了一版觸摸,結果人家一個月后自己就整好了一個更好的方案,我還一直沒有注意到,慚愧。。。
不過項目緊急,有現成的輪子直接拿來用總是好事,重改適配層,這次只需要把ts_read_mt讀到的事件封裝成touch_event事件直接丟給隊列即可:
#define CT_MAX_TOUCH 10
static multi_touch_point_event_t s_points[CT_MAX_TOUCH];ret_t my_main_loop_post_touch_event(main_loop_t* l, event_type_t event_type, touch_event_t* event) {event_queue_req_t r;touch_event_t evt;main_loop_simple_t* loop = (main_loop_simple_t*)l;memset(&r, 0x00, sizeof(r));memset(&evt, 0x00, sizeof(multi_gesture_event_t));return_value_if_fail(loop != NULL, RET_BAD_PARAMS);memcpy(&evt, event, sizeof(touch_event_t));evt.e.type = event_type;evt.e.time = time_now_ms();evt.e.size = sizeof(touch_event_t);r.touch_event = evt;return main_loop_queue_event(l, &r);
}static ret_t tslib_dispatch_one_event(run_info_t* info) {....// note: down, up only trigger once(ET), move will be keep trigger(LT)for (int i = 0; i < max_slots; i++) {if (!(info->samp_mt[0][i].valid & TSLIB_MT_VALID)){continue;}if(info->samp_mt[0][i].pressure > 0){down_number++;if(s_points[i].type == 0 || s_points[i].type == EVT_TOUCH_UP){s_points[i].type = EVT_TOUCH_DOWN;}else{s_points[i].type = EVT_TOUCH_MOVE;}} else{s_points[i].type = EVT_TOUCH_UP;}s_points[i].finger_id = info->samp_mt[0][i].slot;s_points[i].x = info->samp_mt[0][i].x;s_points[i].y = info->samp_mt[0][i].y;event_number++;main_loop_t *loop = (main_loop_t*)info->dispatch_ctx;touch_event_t touch_event;touch_event_init(&touch_event, s_points[i].type, NULL, 0, info->samp_mt[0][i].slot, info->samp_mt[0][i].x / (double)info->max_x, info->samp_mt[0][i].y / (double)info->max_y, info->samp_mt[0][i].pressure);my_main_loop_post_touch_event(loop, s_points[i].type, &touch_event);...
觸點丟失事件
樂極生悲,接下來的一個坑卡了一周都無法解決, 發現應用程序一旦負擔比較高,比如渲染復雜圖形或者事件處理極為頻繁的時候,touch_up的事件經常會被打斷。最后問了微信群,才知道是awtk的活動隊列處理不過來導致丟事件,把main_loop_simple.h的MAIN_LOOP_QUEUE_SIZE宏從默認的20調高到100,重新編譯awtk-linux-fb和應用程序后,問題解決。
對于之前的那個測試demo, 它管理了一個觸點對象數組s_fingers_status
,一旦檢測到手指按下就會生成一個finger_status_t
類型的觸點對象并加入到s_fingers_status 中,finger_status_t
內部也維護了一個point數組,在手指移動的時候就會將手指移動的即時坐標數據加入到數組中,用于on_paint事件的畫線測試,然后在手指彈起時,該觸點對象會被從s_fingers_status中移除并銷毀,如果touch_up事件不上報的話這個事件就會一直留在s_fingers_status里面,導致無法正常獲取正確的按下手指數量,出問題的手指對象也會一直停留在move的狀態,無法恢復。
on_touch_up: 3 size=0
on_touch_down : 1 size=1 460 103
on_touch_down : 3 size=2 591 214
on_touch_move: 1 size=2 459 105
on_touch_move: 3 size=2 591 214
on_touch_move: 1 size=2 459 112
on_touch_move: 3 size=2 591 219
on_touch_move: 1 size=2 458 121
on_touch_move: 3 size=2 590 225
on_touch_move: 1 size=2 458 139
on_touch_move: 3 size=2 589 237
on_touch_move: 1 size=2 459 162
on_touch_move: 3 size=2 587 251
on_touch_down : 2 size=3 295 112
on_touch_move: 2 size=3 295 113
on_touch_move: 2 size=3 296 118
on_touch_move: 2 size=3 298 125
on_touch_move: 2 size=3 302 143
on_touch_move: 1 size=3 461 193
on_touch_move: 2 size=3 308 166
on_touch_move: 3 size=3 585 272
on_touch_move: 1 size=3 464 228
on_touch_move: 2 size=3 318 200
on_touch_move: 3 size=3 582 296
on_touch_down : 0 size=4 137 279
on_touch_move: 0 size=4 137 280
on_touch_move: 0 size=4 144 294
on_touch_move: 0 size=4 154 312
on_touch_move: 0 size=4 173 348
on_touch_move: 0 size=4 195 394
on_touch_move: 1 size=4 468 268
on_touch_move: 1 size=4 493 388
on_touch_move: 2 size=4 378 391
on_touch_move: 1 size=4 506 420
on_touch_up: 2 size=3
on_touch_move: 1 size=3 518 447
on_touch_move: 1 size=3 525 466
on_touch_move: 1 size=3 525 481
on_touch_up: 1 size=2
之前沒找到問題根源的時候就隱約覺得問題跟機器的性能有關,因為測試demo在公司的開發板上兩指到五指都正常,但是到了自己住處, 用百問網的imx6ull開發板一測試很快就出現丟點問題,后面又在適配層嘗試限制touch_move事件的頻率,改為30ms上報一次touch_move事件,發現有一定的降低丟點概率的效果(手指一多還是會丟點),但是仍舊未往事件隊列本身的處理能力去設想,更沒有想到在awtk庫里面可以調這個事件隊列大小,GUI開發還有很多基礎知識需要完善。
最終代碼
tslib_thread.c
/*** File: tslib_thread.c* Author: AWTK Develop Team* Brief: thread to handle touch screen events** Copyright (c) 2018 - 2025 Guangzhou ZHIYUAN Electronics Co.,Ltd.** This program is distributed in the hope that it will be useful,* but WITHOUT ANY WARRANTY; without even the implied warranty of* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the* License file for more details.**//*** History:* ================================================================* 2018-09-07 Li XianJing <xianjimli@hotmail.com> created**/#include <errno.h>
#include <fcntl.h>
#include <unistd.h>
#include "tslib.h"
#include "tkc/mem.h"
#include "tkc/utils.h"
#include "tkc/thread.h"
#include "base/keys.h"#include "tslib_thread.h"
#include <linux/input.h>
#include "multi_gesture.inc"typedef struct _run_info_t {int32_t max_x;int32_t max_y;struct tsdev* ts;void* dispatch_ctx;char* filename;input_dispatch_t dispatch;event_queue_req_t req;struct ts_sample_mt **samp_mt;struct input_absinfo slot;int32_t user_slots;int32_t max_slots;int read_samples;int last_down_number;
} run_info_t;#define RESET "\033[0m"
#define RED "\033[31m"
#define GREEN "\033[32m"
#define BLUE "\033[34m"
#define YELLOW "\033[33m"#define CT_MAX_TOUCH 10
static multi_gesture_touch_points_t* touch_points = NULL;
static multi_touch_point_event_t s_points[CT_MAX_TOUCH];static ret_t tslib_dispatch(run_info_t* info) {ret_t ret = RET_FAIL;char message[MAX_PATH + 1] = {0};// tk_snprintf(message, sizeof(message) - 1, "ts[%s]", info->filename);ret = info->dispatch(info->dispatch_ctx, &(info->req), message);info->req.event.type = EVT_NONE;return ret;
}ret_t my_main_loop_post_touch_event(main_loop_t* l, event_type_t event_type, touch_event_t* event) {event_queue_req_t r;touch_event_t evt;main_loop_simple_t* loop = (main_loop_simple_t*)l;memset(&r, 0x00, sizeof(r));memset(&evt, 0x00, sizeof(multi_gesture_event_t));return_value_if_fail(loop != NULL, RET_BAD_PARAMS);memcpy(&evt, event, sizeof(touch_event_t));evt.e.type = event_type;evt.e.time = time_now_ms();evt.e.size = sizeof(touch_event_t);r.touch_event = evt;return main_loop_queue_event(l, &r);
}static ret_t tslib_dispatch_one_event(run_info_t* info) {int ret = -1;if (info->ts != NULL) {ret = ts_read_mt(info->ts, info->samp_mt, info->max_slots, info->read_samples);}uint8_t event_number = 0, down_number = 0;event_queue_req_t* req = &(info->req);if (ret == 0) {log_warn("%s:%d: get tslib data failed, filename=%s\n", __func__, __LINE__, info->filename);sleep(1);return RET_OK;} else if (ret < 0) {sleep(2);if (access(info->filename, R_OK) == 0) {if (info->ts != NULL) {ts_close(info->ts);}info->ts = ts_open(info->filename, 0);return_value_if_fail(info->ts != NULL, RET_OK);ts_config(info->ts);if (info->ts == NULL) {log_debug("%s:%d: open tslib failed, filename=%s\n", __func__, __LINE__, info->filename);perror("print tslib: ");} else {log_debug("%s:%d: open tslib successful, filename=%s\n", __func__, __LINE__,info->filename);}}return RET_OK;}int max_slots = info->max_slots;// note: down, up only trigger once(ET), move will be keep trigger(LT)for (int i = 0; i < max_slots; i++) {if (!(info->samp_mt[0][i].valid & TSLIB_MT_VALID)){continue;}// printf(YELLOW "BSP sample %d - %ld.%06ld -" RESET " (slot %d) %6d %6d %6d\n",// 0,// info->samp_mt[0][i].tv.tv_sec,// info->samp_mt[0][i].tv.tv_usec,// info->samp_mt[0][i].slot,// info->samp_mt[0][i].x,// info->samp_mt[0][i].y,// info->samp_mt[0][i].pressure);if(info->samp_mt[0][i].pressure > 0){down_number++;if(s_points[i].type == 0 || s_points[i].type == EVT_TOUCH_UP){s_points[i].type = EVT_TOUCH_DOWN;}else{s_points[i].type = EVT_TOUCH_MOVE;}} else{s_points[i].type = EVT_TOUCH_UP;}s_points[i].finger_id = info->samp_mt[0][i].slot;s_points[i].x = info->samp_mt[0][i].x;s_points[i].y = info->samp_mt[0][i].y;event_number++;main_loop_t *loop = (main_loop_t*)info->dispatch_ctx;touch_event_t touch_event;touch_event_init(&touch_event, s_points[i].type, NULL, 0, info->samp_mt[0][i].slot, info->samp_mt[0][i].x / (double)info->max_x, info->samp_mt[0][i].y / (double)info->max_y, info->samp_mt[0][i].pressure);my_main_loop_post_touch_event(loop, s_points[i].type, &touch_event);// print debug// char *msg = "down";// if(s_points[i].type == EVT_TOUCH_MOVE){// msg = "move";// }// else if(s_points[i].type == EVT_TOUCH_UP){// msg = "up";// }// printf("slot %d %s at (%d %d) press %d\r\n", // s_points[i].finger_id, // msg,// s_points[i].x, // s_points[i].y, // info->samp_mt[0][i].pressure); }// printf("down number: %d\r\n", down_number);if(event_number == 1 && (info->last_down_number == 0 || info->last_down_number == 1)){struct ts_sample_mt e = info->samp_mt[0][0];req->event.type = EVT_NONE;req->pointer_event.x = e.x;req->pointer_event.y = e.y;// log_debug("%s%d: e.pressure=%d x=%d y=%d ret=%d\n", __func__, __LINE__, e.pressure, e.x, e.y,// ret);if (e.pressure > 0) {if (req->pointer_event.pressed) {req->event.type = EVT_POINTER_MOVE;} else {req->event.type = EVT_POINTER_DOWN;req->pointer_event.pressed = TRUE;}} else {if (req->pointer_event.pressed) {req->event.type = EVT_POINTER_UP;}req->pointer_event.pressed = FALSE;}info->last_down_number = down_number;return tslib_dispatch(info);}info->last_down_number = down_number;return 0;
}void* tslib_run(void* ctx) {run_info_t info = *(run_info_t*)ctx;if (info.ts == NULL) {log_debug("%s:%d: open tslib failed, filename=%s\n", __func__, __LINE__, info.filename);} else {log_debug("%s:%d: open tslib successful, filename=%s\n", __func__, __LINE__, info.filename);}TKMEM_FREE(ctx);while (tslib_dispatch_one_event(&info) == RET_OK) {};ts_close(info.ts);TKMEM_FREE(info.filename);return NULL;
}static run_info_t* info_dup(run_info_t* info) {run_info_t* new_info = TKMEM_ZALLOC(run_info_t);*new_info = *info;return new_info;
}tk_thread_t* tslib_thread_run(const char* filename, input_dispatch_t dispatch, void* ctx,int32_t max_x, int32_t max_y) {run_info_t info;tk_thread_t* thread = NULL;return_value_if_fail(filename != NULL && dispatch != NULL, NULL);memset(&info, 0x00, sizeof(info));info.max_x = max_x;info.max_y = max_y;info.dispatch_ctx = ctx;info.dispatch = dispatch;info.ts = ts_open(filename, 0);info.filename = tk_strdup(filename);if (info.ts != NULL) {ts_config(info.ts);}
///struct tsdev *ts = info.ts;info.read_samples = 1;info.max_slots = CT_MAX_TOUCH;printf("max_x %d max_y %d TOUCH MAX SLOT=%d\r\n", info.max_x, info.max_y, info.max_slots);info.samp_mt = malloc(info.read_samples * sizeof(struct ts_sample_mt *));if (!info.samp_mt) {printf("create samp_mt failed\r\n");ts_close(ts);return NULL;}for (int i = 0; i < info.read_samples; i++) {info.samp_mt[i] = calloc(info.max_slots, sizeof(struct ts_sample_mt));if (!info.samp_mt[i]) {printf("create samp_mt[%d] failed\r\n", i);for (i--; i >= 0; i--)free(info.samp_mt[i]);free(info.samp_mt);ts_close(ts);return NULL;}}/* 創建不可識別手指類型的多點觸控句柄 */touch_points = multi_gesture_touch_points_create(15);memset(s_points, 0x0, sizeof(multi_touch_point_event_t) * CT_MAX_TOUCH);
/thread = tk_thread_create(tslib_run, info_dup(&info));if (thread != NULL) {tk_thread_start(thread);} else {multi_gesture_gesture_touch_points_destroy(touch_points);if (info.samp_mt) {for (int i = 0; i < info.read_samples; i++) {free(info.samp_mt[i]);}free(info.samp_mt);}ts_close(info.ts);TKMEM_FREE(info.filename);}return thread;
}
練習demo地址
https://gitee.com/tracker647/awtk-practice/tree/master/MapImageTouchZoomTest
為了記錄做業務學到的東西,我額外寫了個demo來演示手指縮放和旋轉的操作,采用vgcanvas矢量畫布來實現旋轉和縮放的效果,原本我想直接從網上下載個地圖圖片然后用vgcanvas_draw_image繪制,來表示地圖的旋轉和縮放,但是不知道是不是imx6ull本身圖形性能太拉,一在on_paint函數調用vgcanvas_draw_image事件整個應用fps就會下降至1,根本無法正常操作圖片,最后只好放棄,改成畫一個紅色矩形來演示效果: