創建外部快照

by Arjun Krishna Babu

通過Arjun Krishna Babu

快照事件：現在如何僅通過拍照即可創建日歷事件 (Snap Event: How you can now create calendar events just by taking a picture)

Google just published my first Android app, Snap Event, in their Play Store. Snap Event creates calendar events from photographs of the event’s poster.

Google剛剛在他們的Play商店中發布了我的第一個Android應用Snap Event 。 Snap Event根據事件海報的照片創建日歷事件。

The app is by no means perfect, but it’s functional and is a proof of concept that the idea works.

該應用程序絕不是完美的，但是它是功能性的，并且是該想法行之有效的概念證明。

This article is a write-up of how I built and published this app.

本文是我如何構建和發布此應用程序的文章。

動機 (Motivation)

People say the best way to learn a new programming language or framework is to do a project with it. Sure, you need some familiarity with the basics of whatever it is that you’re trying to learn. But past that stage, projects are the way to go.

人們說學習一種新的編程語言或框架的最好方法是用它來做一個項目。 當然，您需要熟悉所嘗試學習的基礎知識。 但是經過那個階段，項目才是路要走。

There were three motivating factors behind the development of this app:

此應用程序的開發背后有三個激勵因素：

1. My previous attempt at learning Android two years ago had failed. I wanted to give it another shot.
   我兩年前以前嘗試學習Android的嘗試失敗了。 我想再試一試。
2. I already had a project idea in mind.
   我已經有了一個項目構想。
3. One of my professors asked us each to build an Android app. We’d get credit for it.
   我的一位教授要求我們每個人都構建一個Android應用程序。 我們會為此而獲得榮譽。

Whenever I saw a poster for some event, such as a concert or conference, I took its picture to keep track of what exactly was on the poster. Most people I knew did this. I searched, but I couldn’t find an app that converts images of posters into calendar events. This was surprising. Or maybe I did not search well enough.

每當我看到某事件的海報時，例如音樂會或會議，我都會拍張照片來跟蹤海報上的確切內容。 我認識的大多數人都這樣做。 我進行了搜索，但找不到將海報圖像轉換為日歷事件的應用程序。 這真是令人驚訝。 也許我的搜索不夠好。

So my friend Alexander “Alex” Kaberlein and I decided to build this app.

因此，我和我的朋友Alexander“ Alex” Kaberlein決定開發此應用。

應用程式開發 (App Development)

Creating calendar events from their posters comes down to three things:

從海報創建日歷事件可以歸結為三件事：

1. Detect text from the image.
   從圖像中檢測文本。
2. Make sense of the detected text.
   使檢測到的文本有意義。
3. Create calendar event.
   創建日歷事件。

Our focus was to ship a working app as early as possible, and then iron out its shortcomings. As a consequence, we made some compromises during the design phase. This was supposed to be a learning experience, after all. We have zero intentions of making money from this app.

我們的重點是盡早發布可運行的應用程序，然后消除其缺點。 結果，我們在設計階段做出了一些妥協。 畢竟，這應該是一次學習經歷。 我們從這個應用程序賺錢的意圖零。

從圖像中檢測文本 (Detecting Text From The Image)

Though in retrospect step two was harder, our project could’ve been shelved right away if we had failed to detect text from the image. I had no knowledge whatsoever about computer vision. Alex had some familiarity with OpenCV, but we did not have enough time to come up with our own image recognition models.

盡管回想起來第二步要難一些，但是如果我們未能從圖像中檢測到文本，我們的項目可能會立即被擱置。 我對計算機視覺一無所知。 Alex對OpenCV有一定的了解，但是我們沒有足夠的時間來提出自己的圖像識別模型。

Based on what I read on the internet, I was aware of certain off-the-shelf libraries and services for image recognition. In particular, the three services we considered were:

根據我在互聯網上閱讀的內容，我了解到某些現成的圖像識別庫和服務。 特別是，我們考慮的三項服務是：

1. Amazon Rekognition

   亞馬遜認可

2. Google Cloud Vision

   Google Cloud Vision

3. Google Mobile Vision

   Google移動視覺

It didn’t take long for us to zero in on Google Mobile Vision:

我們花了很長時間將Google Mobile Vision歸零：

• Amazon Rekognition was supposed to be pretty good at detecting objects within the image. But we did not know how well it worked with text.
  亞馬遜Rekognition應該能夠很好地檢測圖像中的物體。 但是我們不知道它與文本的配合情況。
• Cloud Vision was capable of detecting text, but it’s not free (despite being inexpensive).
  Cloud Vision能夠檢測文本，但是它不是免費的(盡管價格便宜)。
• Mobile Vision was capable of detecting text, free, and likely worked well with Android.
  Mobile Vision能夠免費檢測文本，并且可能與Android兼容。

So we chose Mobile Vision.

因此，我們選擇了Mobile Vision。

Next up, we wanted to be sure that Mobile Vision detected text to our satisfaction. We didn’t want to discover at the last moment if Mobile Vision wasn’t up to the task. For this, we built a small toy-app to detect text from images we hard-coded into the app. You can find it here.

接下來，我們希望確保Mobile Vision能夠檢測到令人滿意的文本。 我們不想在最后一刻發現Mobile Vision是否無法完成任務。 為此，我們構建了一個小型玩具應用程序，以從我們硬編碼到應用程序的圖像中檢測文本。 你可以在這里找到它。

The prototype worked well, and we decided to proceed with the main app.

原型運行良好，我們決定繼續使用主應用程序。

Capturing the image and saving it on the phone was easier than I thought. However, I wanted to display all the images taken by the app in vertically scrolling cards. That’s where I ran into my first major set of problems:

捕獲圖像并將其保存在手機上比我想象的要容易。 但是，我想在垂直滾動卡中顯示該應用程序拍攝的所有圖像。 那是我遇到的第一個主要問題集的地方：

• The high-resolution images take up a lot of memory, causing the scrolls to jitter. This was despite using RecyclerView.

  高分辨率圖像占用大量內存，從而導致滾動抖動。 盡管使用了RecyclerView 。

• Centering images within the cards was harder than I thought. You have to fiddle around with a Bitmap object and make non-trivial mathematical calculations.

  在卡中居中放置圖像比我想象的要難。 您必須擺弄Bitmap對象并進行非平凡的數學計算。

The Android documentation (precisely this page) pointed me to solutions for both these problems. There is a library called Glide which handles the complexities associated with pulling multiple images into your app. It also handles annoyances like centering your images properly into your ImageView.

Android文檔(恰好是此頁面)為我指出了這兩個問題的解決方案。 有一個名為Glide的庫，用于處理與將多個圖像拖入您的應用程序相關的復雜性。 它還可以處理將圖像正確居中到ImageView煩惱。

Documentation for Glide could have been better. I had to resort to 3rd party websites to figure out certain use-cases. Speaking of which, you might want to read this excellent introduction to Glide.

Glide的文檔本來可以更好。 我不得不求助于第3方網站來確定某些用例。 說到這，您可能想要閱讀有關Glide的出色介紹。

使用Mobile Vision進行文本識別 (Text Recognition using Mobile Vision)

Mobile Vision supports:

Mobile Vision支持：

1. Face detection

   人臉檢測

2. Barcode detection

   條碼檢測

3. Text detection, which is what we wanted.

   文本檢測 ，這是我們想要的。

The library is capable of providing us text in whichever format we desire. It can be entire blocks of text, lines, words, and so on. Our plan of attack was to extract every single line of text in the poster, and then figure out what each of those lines meant.

該庫能夠為我們提供所需格式的文本。 它可以是文本，行，單詞等的整個塊。 我們的攻擊計劃是提取海報中每行文本，然后弄清楚每行的含義。

There were a couple of things I noticed about text detection using Mobile Vision:

我注意到有關使用Mobile Vision進行文本檢測的幾件事：

• It does not work well with handwritten text.
  它不適用于手寫文本。
• Extracted lines of text are not always in the top-to-bottom order.
  提取的文本行并不總是從上到下的順序。
• Detected text sometimes differs for the same image on multiple runs — an unexpected non-deterministic behavior.
  對于同一圖像，多次運行時檢測到的文本有時會有所不同-意外的不確定性行為。

• It occasionally crashes when using the .jpg format for the images. I resorted to using .png formats.

  使用.jpg格式的圖像時，有時會崩潰。 我求助于使用.png格式。

• Use the ARGV_8888 methodology for storing pixels in memory. It took me hours to figure out why my app kept crashing. Part of my app was using RGB_565 by default, and another part was expecting images in ARGV_8888. See this page to know more about what those configurations mean.

  使用ARGV_8888方法將像素存儲在內存中。 我花了幾個小時才弄清楚為什么我的應用程序持續崩潰。 我的應用程序的一部分默認情況下使用RGB_565 ，而另一部分則期待ARGV_8888圖像。 請參閱此頁面以了解有關那些配置的含義的更多信息。

理解文本 (Making Sense of the Text)

Alex and I concluded that for any calendar event, the three crucial pieces of information are:

我和亞歷克斯得出的結論是，對于任何日歷事件，三個關鍵信息是：

• Schedule
  時間表
• Title
  標題
• Location
  位置

The problem with dates is that they are written in different ways in different parts of the world. For instance, “05–07–2017” is May 7th in the United States and July 5th in almost every other country. Weird ideas involving tricks with the geographic location of the user crossed my mind, but it didn’t feel right. Besides, in countries like the United States where people from all over the world are present, this idea is not foolproof.

日期的問題在于在世界的不同地方以不同的方式編寫日期。 例如，“ 05-07-2017”在美國是5月7日，在幾乎所有其他國家是7月5日。 涉及到與用戶地理位置有關的技巧的怪異想法在我腦海中浮現，但感覺并不正確。 此外，在像美國這樣的世界各地都有人們聚集的國家，這種想法并非萬無一失。

Due to these complications, we decided to infer the month only when it is written out in full, such as “January” (or its abbreviated form “Jan.”). Though not always true, it is somewhat safe to assume that the date and the year would be on the same line as the month.

由于這些復雜性，我們決定僅在完整寫出月份(例如“ January”(或縮寫為“ Jan.”))時推斷月份。 盡管并非總是如此，但可以肯定地說日期和年份與月份在同一行。

Our methodology for detecting the event title and location is so terrible that I’d rather not talk about it. For a given a line of text, I have no reliable strategy to conclusively determine if it is an event title, location, or some other information.

我們檢測事件標題和位置的方法如此糟糕，以至于我不想談論它。 對于給定的一行文本，我沒有可靠的策略來最終確定它是事件標題，位置還是其他信息。

To compensate for this, we dump every single line of text we detect into the description field of the calendar event.

為了彌補這一點，我們將檢測到的每一行文本都轉儲到日歷事件的描述字段中。

創建日歷事件 (Creating the Calendar Event)

A trivial task in the grand scheme of things.

宏偉的事物中的瑣碎任務。

Android stores all calendar events to a central repository called the Calendar Provider. This is why calendar events you create using any calendar app show up on any other calendar app you install. Think of it this way — all media files in your phone would show up on whatever media player you installed.

Android將所有日歷事件存儲到名為Calendar Provider的中央存儲庫中。 這就是您使用任何日歷應用程序創建的日歷事件會顯示在您安裝的任何其他日歷應用程序上的原因。 以這種方式思考-手機中的所有媒體文件都會顯示在您安裝的任何媒體播放器上。

Once all the information needed to create a calendar event is ready, it’s just a matter of starting an activity to create the calendar event.

準備好創建日歷事件所需的所有信息后，只需啟動一個活動來創建日歷事件即可。

We chose to open the calendar app populated with the event information to give users an opportunity to review the event before it is saved. Otherwise, it would silently create the calendar event in the background.

我們選擇打開填充有事件信息的日歷應用程序，以便用戶有機會在保存事件之前對其進行查看。 否則，它將在后臺靜默創建日歷事件。

在Google Play商店上發布 (Publishing on Google Play Store)

Like most other things by Google, the steps for publishing an Android app are well-documented. Once I felt ready to launch the app, I followed the official launch checklist.

與Google的大多數其他事情一樣，發布Android應用程序的步驟也有據可查。 一旦我準備好啟動該應用程序，就遵循官方的啟動清單。

It wasn’t a quick process, though. It took them over 24 hours to verify my one-time registration fee of $25, and 6 more hours for my app to show up on the Google Play store after I clicked the button to rollout and publish the app.

但是，這不是一個快速的過程。 他們花了超過24小時的時間才能確認我的一次性注冊費25美元，并且在我單擊按鈕進行展示和發布該應用后，我的應用又在Google Play商店中展示了6個多小時。

我從開發此應用程序中學到的東西 (What I Learned From Developing This App)

• Basic Android programming. I’m now confident about reading through the documentation (as well as other resources) to figure out how to get things done. Now I can create more complex Android apps in the future.
  基本的Android編程。 我現在有信心閱讀文檔(以及其他資源)以弄清楚如何完成任務。 現在，我可以在將來創建更復雜的Android應用程序。
• How to actually publish the Android app.
  如何實際發布Android應用程序。
• How to use the Glide library to display multiple images efficiently.
  如何使用Glide庫有效顯示多個圖像。
• How to set up and use Google’s Mobile Vision library.
  如何設置和使用Google的Mobile Vision庫。

缺點 (Shortcomings)

I’ve already covered many of the shortcomings of the app in the App Development section above. But I’ll summarize the issues here:

我已經在上面的“應用程序開發”部分中介紹了該應用程序的許多缺點。 但我將在這里總結這些問題：

• Date detection happens only if the date is written out in full.

  僅當日期被完整寫出時，才會進行日期檢測。

  In the real world, dates are not always written out in full. This severely limits the usability of the app.

  在現實世界中，日期不一定總是完整寫出。 這嚴重限制了該應用程序的可用性。

• Date detection fails if it contains ordinal suffixes like “st,” “nd,” “rd,” or “th.” However, this can be quickly fixed.
  如果日期檢測包含序數后綴(如“ st”，“ nd”，“ rd”或“ th”)，則日期檢測將失敗。 但是，可以快速解決此問題。
• Event title and location detection are not perfect. I’m contemplating using machine learning models to accomplish this task, but it’s still quite far away.
  事件標題和位置檢測并不完美。 我正在考慮使用機器學習模型來完成此任務，但是它仍然很遙遠。
• The images are saved onto the phone using a terrible strategy that I’m ashamed to mention here. Even though it works perfectly. This would be evident only if you read through the source-code.
  這些圖片會以一種可怕的策略保存到手機中，對此我感到here愧。 即使效果完美。 僅當您閱讀源代碼時，這才顯而易見。
• User experience can be improved.
  可以改善用戶體驗。

接下來是什么 (What Next)

I’ll fix the relatively easy things I mentioned in the shortcomings section. But I likely won’t pursue this project too much further due to reasons I mention next.

我將修復我在缺點部分中提到的相對簡單的問題。 但是由于接下來要提及的原因，我可能不會對這個項目進行過多的研究。

谷歌鏡頭 (Google Lens)

Among the host of exciting things Google announced during Google I/O 2017 is Google Lens. Google Lens does exactly what our app does (along with a bunch of other cool things). And Google Lens does it rather well, much better than what I have created through Snap Event.

在Google I / O 2017期間Google宣布的令人興奮的事情之一是Google Lens。 Google Lens確實完成了我們的應用程序的工作(以及許多其他很酷的事情)。 而且Google Lens的效果相當好，比我通過Snap Event創建的要好得多。

And I’m happy and excited about the fact that Google did it. At least I expect it to be a lot more reliable and useful.

我為Google做到這一點感到高興和興奮。 至少我希望它更加可靠和有用。

獲取快照事件 (Get Snap Event)

• As mentioned, Snap Event is now available on Google Play.

  如前所述， Snap Event現在可在Google Play上使用。

• The source-code of the app is available on GitHub.

  該應用程序的源代碼可在GitHub上找到。

You can read about more of my projects on my blog.

您可以在我的博客上閱讀有關我的更多項目的信息 。

翻譯自: https://www.freecodecamp.org/news/snap-events-how-you-can-now-create-calendar-events-just-by-taking-a-picture-af21f3bfaeef/

創建外部快照