敏捷數據科學pdf

In this article, we'll discuss how agile principles and values can be applied to the way you approach data science projects.

在本文中，我們將討論如何將敏捷性原則和價值觀應用于您處理數據科學項目的方式。

Project management methodologies are commonly used to get projects done or get a product (often referred to as a tool) produced. They are, in general, processes and frameworks which break down the overall objective to individual tasks organised on a timeline. This can be adapted and used to approach data science projects.

項目管理方法通常用于完成項目或獲得產品(通常稱為工具)。 通常，它們是將總體目標分解為按時間表組織的各個任務的流程和框架。 可以對其進行修改，并用于處理數據科學項目。

In the past, the traditional Waterfall methodology (dated way back to 1970) has been very popular. It defines all requirements and parameters of the product at the start, so that the project team can work towards this target in sequential phases.

過去，傳統的Waterfall方法 (可追溯至1970年)非常流行。 它從一開始就定義了產品的所有要求和參數，以便項目團隊可以在相繼的階段朝著這個目標努力。

This method has been successful in the manufacturing industry where product specifications seldom vary with time. It requires very extensive upfront planning, and ideally, the output product is exactly the same as specified in the beginning.

這種方法在制造規格很少隨時間變化的制造業中很成功。 它需要非常廣泛的前期計劃，理想情況下，輸出產品與開始時指定的產品完全相同。

But the Waterfall methodology started to become unsuitable for software projects. Because of this, many popular project management methodologies have emerged over the years, especially in the software development industry. Let me share the most popular one.

但是瀑布方法論開始變得不適用于軟件項目。 因此，多年來出現了許多流行的項目管理方法 ，尤其是在軟件開發行業中。 讓我分享最受歡迎的一個。

敏捷框架 (Agile Framework)

Agile is a way of working developed in 2001, and is a widely used to manage software development projects. It is suitable for fast-paced development cycles and has provision for changing specifications throughout the design and build process. It is flexible, and strives for iterative incremental improvement in the product through team collaboration. In short, Agile is to plan, build, test, learn, repeat.

敏捷是2001年開發的一種工作方式，被廣泛用于管理軟件開發項目。 它適用于快節奏的開發周期，并可以在整個設計和構建過程中更改規格。 它具有靈活性，并通過團隊協作努力實現產品的迭代增量改進。 簡而言之，敏捷就是計劃，構建，測試，學習，重復。

Agile teams are responsive to the unpredictable requirements as the project unfolds, through iterative work processes. Below are Agile principles which serve as a framework (guideline) to the way of working:

隨著項目的開展，敏捷團隊通過迭代的工作流程對不可預測的需求做出響應。 以下是用作工作方式框架(準則)的敏捷原則 ：

• Customer satisfaction through early and continuous software delivery
  通過盡早連續交付軟件來使客戶滿意
• Accommodate changing requirements throughout the development process
  在整個開發過程中適應不斷變化的需求
• Frequent delivery of working software, as the working software is the primary measure of progress
  頻繁交付工作軟件，因為工作軟件是進度的主要衡量標準
• Collaboration and interaction between the business stakeholders (client) and developers (vendor) throughout the project, including face-to-face communication within the development team
  在整個項目中，業務涉眾(客戶)與開發人員(供應商)之間的協作和互動，包括開發團隊內部的面對面交流
• Support, trust, and motivate the people involved
  支持，信任和激勵相關人員
• Agile frameworks to support a consistent development pace
  敏捷框架支持一致的發展速度
• Attention to technical detail and design enhances agility
  注重技術細節和設計可增強敏捷性
• Simplicity in looking for solutions
  尋找解決方案的簡便性
• Regular reflections in the self-organising team on how to become more effective
  自組織團隊定期思考如何提高效率

Agile projects are characterized by a series of tasks that are conceived, executed and adapted as the situation demands. However, Agile focus is not on what to do, but how to think. Agile values and places priority on:

敏捷項目的特點是根據情況需要構思，執行和調整一系列任務。 但是，敏捷的重點不是做什么，而是如何思考 。 敏捷價值觀和重中之重 上：

• Individuals and interactions (rather than processes and tools)
  個人和互動(而不是流程和工具)
• Working software (rather than comprehensive documentation)
  工作軟件(而不是全面的文檔)
• Customer collaboration (rather than contract negotiation)
  客戶協作(而不是合同談判)
• Response to change (rather than following a predefined rigid plan)
  對變化的響應(而不是遵循預先定義的嚴格計劃)

敏捷實踐與數據科學 (Agile practices and Data Science)

While Agile principles and priorities are employed for greater productivity, most of them can be leveraged for data science (DS) projects.

盡管采用敏捷原則和優先級來提高生產率，但大多數原理和優先級可用于數據科學(DS)項目。

Moreover, data scientists do not know how to schedule the project because it is impossible to determine a specific timeline for the type of “research” and exploratory work. Most DS projects require trial and error by going down different paths and trying different techniques. They do not have an element of certainty in the output, so Agile can be used to direct the workflow.

此外，數據科學家不知道如何安排項目時間，因為無法確定“研究”和探索性工作的具體時間表。 大多數DS項目需要走不同的道路并嘗試不同的技術，因此需要反復試驗。 它們在輸出中沒有確定性，因此可以使用敏捷來指導工作流程。

Most other projects deal with what customers want, what the developers want, and what the business seeks. When working with DS, another perspective is added: what the data is telling you.

其他大多數項目都處理客戶的需求，開發人員的需求以及企業的需求。 使用DS時，添加了另一個角度： 數據告訴您什么 。

Data scientists cannot make any sense out of the data unless they develop a basic understanding of it. There is a lot of investigation, exploration, testing and tuning. Agile uses the concept of iteration and constant feedback in order to refine a system under development, in order to move up the Data-Value Pyramid.

除非他們對數據有基本的了解，否則數據科學家無法從數據中獲得任何意義。 有很多調查，探索，測試和調整。 敏捷使用迭代和恒定反饋的概念來優化正在開發的系統，以提升Data-Value Pyramid 。

When working on DS projects, insights are not immediately achievable. Multiple iterations are needed before any insights can be discovered.

在DS項目上工作時，無法立即獲得見解。 在發現任何見解之前，需要多次迭代。

如何應用敏捷實踐 (How agile practices can be applied)

I will explain the main Agile working practices (Scrum framework), and how they can be applied to DS:

我將解釋主要的敏捷工作實踐( Scrum框架 )，以及如何將其應用于DS：

Define the business need and the project objective. This is usually driven by the product owner who is responsible for the product features and quality. It is the big picture stuff, but this is the core belief that you will refer back to as you build.

定義業務需求和項目目標 。 這通常是由負責產品功能和質量的產品所有者驅動的。 這是全局的東西，但這是您在構建時會引用的核心信念。

In DS, the product owner could be the client, the business, or the end customer (for example, end user of a prediction tool). Understand what problems the product owner is facing and tailor the project proposal to meet their needs.

在DS中，產品所有者可以是客戶，企業或最終客戶(例如，預測工具的最終用戶)。 了解產品負責人面臨的問題，并定制項目建議書以滿足他們的需求。

Build the backlog. Focusing on the user requirements (“user stories” in Agile), a list of tasks is derived that you need to accomplish to build product features or improve product performance.

建立積壓 。 著眼于用戶需求(敏捷中的“用戶案例”)，將導出您需要完成的任務列表，以構建產品功能或提高產品性能。

The DS team builds the backlog together with the product owner to determine the product features and performance targets. The backlog could start from getting the data in the structured way before they can be analysed. Then it could be a list for feature selection or feature engineering, or a list of models to select, tune and optimise.

DS團隊與產品所有者一起構建積壓訂單，以確定產品功能和性能目標。 積壓工作可以從以結構化方式獲取數據開始，然后再對其進行分析。 然后可能是用于特征選擇或特征工程的列表，或者是要選擇，調整和優化的模型的列表。

Prioritise the backlog, identify the backlog tasks which will bring the most value with the least effort.

優先安排待辦事項 ，確定將以最少的努力帶來最大價值的待辦事項。

In DS, not every approach is worth trying, so cover the most promising ones first. When the main ones are conveyed, you might find that the remaining others are not as important as initially thought.

在DS中，并非每種方法都值得嘗試，因此請首先涵蓋最有前途的方法。 當傳達主要的內容時，您可能會發現其余的內容并不像最初想象的那么重要。

Do a sprint (the actual development work). Sprints are usually two-weeks cycles where high priority tasks on the backlog are worked on.

進行沖刺 (實際開發工作)。 沖刺通常是兩個星期的周期，其中需要處理積壓的高優先級任務。

In DS, each sprint could be two to four weeks depending on the team size. During the sprint, always complete the task with the highest priority before moving on to the next in line.

在DS中，每個沖刺可能需要兩到四個星期，具體取決于團隊規模。 在sprint期間，請始終以最高優先級完成任務，然后再繼續進行下一行。

Have daily standups. Standup meetings are for team members to be accountable to one another on their progress in the current sprint. Each team member take turns reporting their status — what was done the day before, what to do today, any potential obstacles. The most effective communication happens when DS team members meet face-to-face to share their work.

每天站起來 。 站立會議的目的是使團隊成員對當前沖刺中的進度互相負責。 每個團隊成員輪流報告其狀態-前一天做什么，今天該做什么以及任何潛在的障礙。 當DS團隊成員面對面分享他們的工作時，最有效的溝通發生了。

Review the sprint output (sprint retrospective meeting). At the end of two weeks, there should be a functional output for the project team to demonstrate, with an incremental improvement in the product.

查看sprint輸出 (sprint回顧會議)。 在兩周末，應該有一個功能輸出供項目團隊演示，并在產品上進行逐步改進。

Data scientists should share the outputs before trying to perfect the processes. Get feedback from client stakeholders and prepare for the next sprint. Regular feedback is a key principle for the Agile way of iterative incremental improvement.

數據科學家應在嘗試完善流程之前共享輸出。 獲得客戶利益相關者的反饋，為下一個沖刺做準備。 定期反饋是迭代改進增量的敏捷方法的關鍵原則。

Prepare for the next sprint. Identify the tasks that are going well and keep doing them, and identify those that are impediments to be removed.

準備下一個沖刺 。 確定進展順利的任務并繼續執行，并找出要消除的障礙。

It is important to understand that, unlike software development, DS is more experiment-based than task-based. DS helps explore data so it should be treated as multiple research experiments. Once again, build and prioritise the backlog so that the next sprint can be carried out, to work on the next improvement areas.

重要的是要理解，與軟件開發不同，DS更加基于實驗而不是基于任務。 DS有助于探索數據，因此應將其視為多個研究實驗。 再次構建積壓工作并確定積壓工作的優先級，以便可以進行下一個沖刺，以進行下一個改進領域。

Roll out the final product. When all stakeholders agree that no more improvement is needed in the product, it is ready for the final deployment.

推出最終產品 。 當所有利益相關者同意不再需要產品改進時，就可以進行最終部署了。

DS projects follow the “law of diminishing improvement”. For example, if a model has achieved 70% accuracy, the next 5–10% improvement will take a lot more effort than before, and it also depends on the limitations in the data set. Decide in the team whether the efforts are worth the incremental improvement.

DS項目遵循“遞減改進法則”。 例如，如果模型已達到70％的準確度，那么下一個5–10％的改進將比以前花費更多的精力，并且它還取決于數據集的限制。 確定團隊中的努力是否值得進行逐步改進。

與客戶的挑戰 (Challenges with the client)

Besides having adequate communication between the DS team and the client, the client’s expectations have to be managed.

除了在DS團隊和客戶之間進行充分的溝通外，還必須管理客戶的期望。

All clients generally love the idea that Agile is flexible, and that it grants them more opportunities to change their mind as the project develops. However, they might not realise that such flexibility is also costly in both time and money. Here are some things you should do:

所有客戶通常都喜歡敏捷性是靈活的想法，并且隨著項目的發展，它為他們提供了更多改變主意的機會。 但是，他們可能沒有意識到這種靈活性在時間和金錢上都是昂貴的。 這是您應該做的一些事情：

靈活性成本 (The cost of flexibility)

Get the client to understand that flexibility is inevitably expensive. It is like how a flexible full-fare economy ticket which allows itinerary changes will cost much more than the fixed one. Making changes also means that the client is paying for past wasted time and effort.

讓客戶了解靈活性是不可避免的 。 這就像一張靈活的，可以更改行程的全票價經濟艙機票，其價格將比固定票價高得多。 進行更改還意味著客戶要為過去浪費的時間和精力付費。

設定期望 (Set expectations)

Set the client’s expectation to commit time for frequent sprint retrospective meetings (e.g. every two weeks) to evaluate the completed sprints.

設定客戶的期望，以花時間參加頻繁的沖刺回顧會議 (例如，每兩周一次)，以評估完成的沖刺。

On top of that, the client representative in each meeting needs to be (empowered by higher management) able to make decisions on product specifications. For Agile to work, the client needs to provide continuous feedback and priority setting to keep the project moving.

最重要的是，每次會議的客戶代表都必須(由高級管理層授權 )能夠做出有關產品規格的決策。 為了使敏捷工作，客戶需要提供連續的反饋和優先級設置，以保持項目的進展。

信任很重要 (Trust is important)

Earn the client’s trust and show them that each iteration is done with the best possible efforts to deliver value and improve the product.

贏得客戶的信任，并向他們表明，每次迭代都是盡最大的努力來交付價值和改進產品。

While holding the decision making power, the client also expects an iteration to have tremendous improvement.

客戶在擁有決策權的同時，還希望迭代能夠帶來巨大的改進。

Such imbalance in responsibility in the client-vendor relationship should be converted to mutual trust and willingness to experiment together. Agile’s principle in collaboration means it is a team effort in both making decisions and delivering value.

客戶與供應商關系中責任的這種不平衡應轉化為相互信任和愿意共同試驗的意愿。 敏捷的協作原則意味著這是團隊在決策和交付價值上的努力。

最低可行產品 (Minimum Viable Product)

One key feature of the Agile way of working is the development of a minimum viable product (MVP). This is the most fundamental configuration of the product (or tool).

敏捷工作方式的一個關鍵特征是開發最小可行產品( MVP )。 這是產品 (或工具)的最基本配置 。

After the project objectives have been defined, the team makes a proposal regarding the approach to the problem. This includes building the MVP within the shortest possible time (like one month for DS projects). The MVP has only the most important functionalities, but its performance may not be the most optimal.

定義項目目標后，團隊將就解決問題的方法提出建議。 這包括在最短的時間內(例如DS項目一個月)構建MVP。 MVP僅具有最重要的功能，但其性能可能并非最佳。

This might seem very risky – putting a less-than-finished version up for the client to test. So the team (including the client) has to be prepared for it. The purpose is to make the MVP work, test it, and see if it is really going in the correct direction of solving the problem and helping the business case.

這似乎很有風險-放置一個未完成的版本供客戶端測試。 因此，團隊(包括客戶)必須為此做好準備。 目的是使MVP正常工作，對其進行測試，并查看它是否真的朝著解決問題和幫助業務案例的正確方向發展。

The MVP will grow better, because the DS team is going to use what they have learnt from the MVP feedback to build an improved version. Agile is about continuously deploying and learning from your mistakes, and working with the client to make the product better.

MDS將會變得更好，因為DS團隊將使用他們從MVP反饋中學到的知識來構建改進版本。 敏捷是指不斷地從錯誤中進行部署和學習，并與客戶合作以使產品更好。

Agile is to plan, build, test, learn, repeat.

敏捷就是計劃，構建，測試，學習，重復。

DS項目可交付成果 (DS project deliverable)

The Agile way of working allows data scientists the ability to prioritize and create roadmaps based on requirements and goals. With each iteration, data scientists can learn something new, get more refined results, and ride on them for the next incremental improvement.

敏捷的工作方式使數據科學家能夠根據需求和目標確定優先級并創建路線圖。 每次迭代，數據科學家都可以學習新知識，獲得更完善的結果，并利用它們進行下一次增量改進。

Below are some Agile project deliverables to shape and guide project process:

以下是一些敏捷項目可交付成果，用于塑造和指導項目流程：

• Project vision statement: A summary that articulates the goals for the project.

  項目愿景聲明 ：概述項目目標的摘要。

• Project roadmap: The high-level view of the requirements needed to achieve the project vision.

  項目路線圖 ：實現項目遠景所需需求的高級視圖。

• Project backlog: Ordered by priority, this is the full list of what is needed to support your project.

  項目積壓 ：按優先級排序，這是支持您的項目所需的完整列表。

• Release plan: A timetable for the release of a working product (or tool), but not documentation. Projects should be self-documenting along the way.

  發布計劃 ：發布有效產品(或工具)的時間表，而不是文檔。 在此過程中，項目應該是自我記錄的。

• Sprint backlog: The user stories (requirements), goals, and tasks linked to the current sprint.

  Sprint積壓 ：與當前Sprint鏈接的用戶案例(要求)，目標和任務。

• Increment: The working product functionality that is presented to the stakeholders at the end of the sprint and could potentially be given to the client. The goal is not to deliver more but to get a higher value output.

  增量 ：在sprint結束時提供給涉眾的工作產品功能，并且可以潛在地提供給客戶。 目標不是提供更多，而是獲得更高的價值 。

摘要 (Summary)

Agile is going to be adopted by more DS project teams in the near future. Many data scientists have reported that it makes them more productive.

在不久的將來，更多的DS項目團隊將采用敏捷。 許多數據科學家報告說，這使它們更具生產力。

This is not because the data scientists have become more skilled, but because Agile can help them optimize their projects. Instead of spending time on models that are unlikely to reveal any productive results, it is better to spend that time for other result-driven purposes.

這不是因為數據科學家變得更加熟練，而是因為敏捷可以幫助他們優化項目。 與其將時間花在不可能顯示任何有效結果的模型上，不如將時間花在其他以結果為導向的目的上。

Being “agile” (flexible) means you need to adopt a dynamic approach in planning and be adaptable to the changing needs of the new situation when it arises.

“敏捷”(靈活)意味著您需要在規劃中采用動態方法，并在新情況出現時適應新的變化需求。

The Agile environment appeals to quick action, fail quickly, discuss and evaluate, then try again using a different approach or an improved method. It works great in dynamic environments where there is a potential for changing or evolving requirements.

敏捷環境呼吁采取快速行動，Swift失敗，進行討論和評估，然后使用其他方法或改進的方法再試一次。 它在動態環境中非常有用，因為動態環境中可能會發生變化或不斷變化的需求。

All the best to your DS projects!

祝您的DS項目一切順利！

Reference:Data-science? Agile? Cycles? My method for managing data-science projects in the Hi-tech industry.

參考： 數據科學？ 敏捷？ 周期？ 我在高科技行業中管理數據科學項目的方法。

翻譯自: https://www.freecodecamp.org/news/applying-agile-methodology-to-data-science-projects/

敏捷數據科學pdf