fritz 使用手冊

by Eric Hsiao

蕭敬軒

Fritz對象檢測指南：使用機器學習在Android中構建寵物監控應用 (A guide to Object Detection with Fritz: Build a pet monitoring app in Android with machine learning)

Whether it is detecting plant damage for farmers, tracking vehicles on the road, or monitoring your pets — the applications for object detection are endless. With the rise of mobile frameworks like TensorFlow Lite and Core ML, more and more mobile apps leverage the power of machine learning to create features that leave us in awe.

無論是為農民發現植物受損 ，在路上追蹤車輛還是監視寵物-用于物體檢測的應用層出不窮。 隨著TensorFlow Lite和Core ML等移動框架的興起，越來越多的移動應用利用機器學習的力量來創建令我們敬畏的功能。

那么對象檢測到底是什么呢？ (So what exactly is object detection?)

In plain English, object detection identifies and locates specific items in an image or live video with a bounding box.

用簡單的英語來說，物體檢測可以識別和定位帶有邊框的圖像或實時視頻中的特定項目。

But creating features powered by machine learning isn’t easy. Many engineering teams cannot justify the time and resources. You need the right in-house expertise to collect data, train a model, and iterate on the performance and accuracy. Understandably, with pressure from product teams to deliver value quickly for end-users, potential features are tossed aside in a backlog abyss.

但是創建由機器學習支持的功能并不容易。 許多工程團隊無法證明時間和資源的合理性。 您需要合適的內部專家來收集數據，訓練模型并迭代性能和準確性。 可以理解的是，在產品團隊的壓力下，他們需要Swift為最終用戶交付價值，潛在的功能被積壓的深淵拋棄了。

In this post, I’ll show you how any Android developer can use real-time object detection to create an app that detects and recognizes pets — all in less than 30 minutes. To do this, I’ll use the Fritz SDK (full disclosure, I work at Fritz) which makes it easier to leverage machine learning capabilities without any prior experience.

在這篇文章中，我將向您展示任何Android開發人員如何使用實時對象檢測來創建可檢測和識別寵物的應用程序-只需不到30分鐘即可完成。 為此，我將使用Fritz SDK (完整的披露內容，我在Fritz工作)，這使得在沒有任何先驗經驗的情況下更容易利用機器學習功能。

入門 (Getting started)

To start using the Fritz SDK, we’ll go through adding the necessary dependencies in a sample app that we’ve created.

要開始使用Fritz SDK，我們將在我們創建的示例應用程序中添加必要的依賴項。

1.首先，創建一個Fritz帳戶 (1. First, create a Fritz account)

Sign up here and follow the get started directions.

在這里注冊并按照入門指南進行操作 。

2.克隆示例相機應用 (2. Clone the sample camera app)

Set up a skeleton app that includes a video feed and camera code. In this tutorial, we won’t go depth into the Camera 2 API, but if you have any questions, please leave a comment.

設置一個包含視頻供稿和攝像機代碼的骨架應用程序。 在本教程中，我們不會深入研究Camera 2 API ，但是如果您有任何疑問，請發表評論。

git clone https://github.com/fritzlabs/camera-sample-app

3.在webapp中注冊Android應用 (3. Register the Android app in the webapp)

You need to register your app with Fritz in order to use ML-features. When you’re adding the app to Fritz, use the same applicationId (ai.fritz.camera) as the app/build.gradle.

您需要向Fritz注冊應用才能使用ML功能。 將應用程序添加到Fritz時，請使用與app / build.gradle相同的applicationId(ai.fritz.camera) 。

Make sure you note the API Key for Step 5. If you need to access it again, you can go to Project Settings > Your App (Pet Monitor)> Show API Key (in the options menu).

確保注意步驟5的API密鑰。如果需要再次訪問它，則可以轉到“項目設置”>“您的應用程序(寵物監控器)”>“顯示API密鑰”(在選項菜單中)。

4.在app / build.gradle中添加FritzCore +依賴項 (4. Add FritzCore + dependencies in app/build.gradle)

Make sure to add the Fritz repository. This will allow you to download the necessary dependencies:

確保添加Fritz存儲庫。 這將允許您下載必要的依賴項：

In the dependencies section, add these 2 libraries:

在“依賴關系”部分中，添加以下兩個庫：

dependencies {    implementation "ai.fritz:core:2.0.0"    implementation "ai.fritz:vision-object:2.0.0"}

5.配置SDK (5. Configure the SDK)

Call Fritz.configure in the Application or MainActivity onCreate lifecycle method with the API key you got in Step 3.

使用在步驟3中獲得的API密鑰，在Application或MainActivity onCreate生命周期方法中調用Fritz.configure 。

With that, you’re ready to use object detection in your app.

這樣，您就可以在應用程序中使用對象檢測了。

在實時視頻中檢測貓狗 (Detecting dogs & cats on live video)

Now let’s get to the fun stuff. We’ll jump into the MainActivity and use the object detection predictor on each frame passed in on the video stream.

現在讓我們來看看有趣的東西。 我們將跳入MainActivity，并在視頻流上傳遞的每個幀上使用對象檢測預測值。

1.獲取FritzVisionObjectPredictor的實例 (1. Get an instance of FritzVisionObjectPredictor)

The predictor takes in a FritzVisionImage and returns a list of FritzVisionObjects detected.

預測器接收FritzVisionImage并返回檢測到的FritzVisionObjects列表。

2.將每一幀轉換為FritzVisionImage對象 (2. Convert each frame to a FritzVisionImage object)

Use either fromBitmap or fromMediaImage static methods to create an object from a Bitmap or media.Image object. For the sample app, use fromMediaImage, which also takes in the rotation applied on the image from the camera.

使用fromBitmap或fromMediaImage靜態方法從Bitmap或media.Image對象創建對象。 對于示例應用程序，請使用fromMediaImage ， 這也吸收了應用在相機圖像上的旋轉。

The rotation depends on the device rotation and the camera orientation sensor. The cameraId identifies the active camera being used on the device (front, back, etc.), and you can get the rotation angle with the following helper method.

旋轉取決于設備旋轉和相機方向傳感器。 cameraId標識設備上正在使用的活動攝像機(正面，背面等)，您可以使用以下輔助方法獲取旋轉角度。

int rotation = FritzVisionOrientation.getImageRotationFromCamera(this, cameraId);

Finally, create a FritzVisionImage object with the Image and rotation value.

最后，使用Image和rotation值創建一個FritzVisionImage對象。

3.運行預測 (3. Run prediction)

Pass the image into the predictor to detect different objects in the image.

將圖像傳遞到預測變量中以檢測圖像中的不同對象。

4.過濾結果并顯示邊界框 (4. Filter the result and display bounding boxes)

Each FritzVisionObject comes with a label, a confidence score, and a bounding box that shows where it’s located on the original image. In this case, we only care about pets (specifically cats and dogs), so we can filter out the other items.

每個FritzVisionObject都有一個標簽，一個置信度分數和一個邊界框，用于顯示其在原始圖像上的位置。 在這種情況下，我們只關心寵物(特別是貓和狗)，因此我們可以過濾掉其他物品。

Finally, display the bounding boxes around your pets. FritzVisionObject has a convenient method called drawOnCanvas which makes it easy to display the detected objects.

最后，顯示寵物周圍的邊界框。 FritzVisionObject有一個稱為drawOnCanvas的便捷方法，可輕松顯示檢測到的對象。

Here’s the complete code after the render callback:

這是渲染回調之后的完整代碼：

Notice the scale factor on the boxes. This is because the media.Image object we used to create the FritzVisionImage object is the same size as the preview viewport. In the camera sample app, it’s 1280 x 960. The bounding boxes will have coordinates associated with the preview size. Since we want to show this on the full screen, we need to scale the result to the phone’s viewport.

注意包裝盒上的比例因子。 這是因為我們用來創建FritzVisionImage對象media.Image對象是大小預覽視窗相同。 在相機示例應用中，尺寸為1280 x960。邊界框將具有與預覽尺寸相關聯的坐標。 由于我們要在全屏上顯示此內容，因此需要將結果縮放到手機的視口。

Here’s the final result:

這是最終結果：

For the finished code, take a look at the GitHub repo.

有關完成的代碼，請查看GitHub repo 。

為什么這很有用 (Why this is useful)

With the machine learning feature behind this basic app, there are ton of different features you can create (both practical and goofy):

有了這個基本應用程序背后的機器學習功能，您可以創建許多不同的功能(實用的和愚蠢的)：

• Alert the owners with a text message if the dog walker hasn’t returned.
  如果walk狗沒有回來，請通過短信提醒主人。
• Record a message telling your dog to “Sit down!” when they’re running around the room with no one around. I bet you could capture funny photos of your dog in this moment, too.
  記錄一條消息，告訴您的狗“坐下！” 當他們在房間里跑來跑去時，沒人在附近。 我敢打賭，您也可以在這一刻捕捉狗的有趣照片。
• Show the user a message when a cat / dog is detected (take a look at the completed code for an example)
  在檢測到貓/狗時向用戶顯示一條消息(以完整代碼為例)
• Sound an alarm when the camera detects cats (I’m allergic).
  相機檢測到貓時發出警報聲(我過敏)。

Of course, not many people have a spare Android tablet / phone that they can use as an expensive pet monitoring camera, but this is just a simple example among many different possibilities for how you might create an app with object detection using Fritz. I can’t wait to see what all the creative developers of the world build using object detection.

當然，沒有多少人有備用的Android平板電腦/手機可以用作昂貴的寵物監控攝像頭，但這只是您使用Fritz創建帶有對象檢測的應用程序的許多不同可能性中的一個簡單示例。 我迫不及待地想看看世界上所有的創意開發人員都使用對象檢測構建了什么。

Got an idea? Leave a comment!

有想法嗎？ 發表評論！

I’m a lead developer at Fritz specializing in mobile machine learning. If you’re looking to create features powered by AI/ML, we offer prebuilt APIs (image segmentation, image labeling, style transfer) and custom model support.

我是Fritz的首席開發人員，專門研究移動機器學習。 如果您要創建由AI / ML提供支持的功能，我們提供預構建的API( 圖像分割 ， 圖像標簽 ， 樣式轉換 )和自定義模型支持。

翻譯自: https://www.freecodecamp.org/news/a-guide-to-object-detection-with-fritz-build-a-pet-monitoring-app-in-android-with-machine-learning-a8ed500978e5/

fritz 使用手冊