Python Microservices Development_ Build efficient and lightweight microservices using the Python tooling e.pdf

1、Python Microservices DevelopmentSecond EditionBuild efficient and lightweight microservices using the Python tooling ecosystemSimon FraserTarek ZiadBIRMINGHAMMUMBAIPython Microservices DevelopmentSecond EditionCopyright 2021 Packt PublishingAll rights reserved.No part of this book may be reproduced,

2、stored in a retrieval system,or transmitted in any form or by any means,without the prior written permission of the publisher,except in the case of brief quotations embedded in critical articles or reviews.Every effort has been made in the preparation of this book to ensure the accuracy of the infor

3、mation presented.However,the information contained in this book is sold without warranty,either express or implied.Neither the author(s),nor Packt Publishing or its dealers and distributors,will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

4、Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals.However,Packt Publishing cannot guarantee the accuracy of this information.Producer:Shailesh JainAcquisition Editor Peer Reviews:Saby DSi

5、lvaProject Editor:Rianna RodriguesContent Development Editor:Alex PattersonCopy Editor:Safis EditingTechnical Editor:Karan SonawaneProofreader:Safis EditingIndexer:Rekha NairPresentation Designer:Pranit PadwalFirst published:July 2017Second edition:September 2021Production reference:1280921Published

6、 by Packt Publishing Ltd.Livery Place35 Livery StreetBirminghamB3 2PB,UK.ISBN 978-1-80107-630-ContributorsAbout the authorsSimon Fraser is a Site Reliability Engineer for Cisco Meraki.He has over twenty years of experience in computing,both developing and running systems.He has worked as a Systems A

7、dministrator for an internet service provider;an infrastructure and high-performance computing engineer at the Welcome Sanger Institute,and as a Firefox Release Engineer at Mozilla.He studied Cybernetics and Computer Science at the University of Reading and has also taught academic courses in progra

8、mming for scientists.I would like to thank Emma Apted and Helen Cook for their frequent support and encouragement,ranging from proofreading to miming a book opening when Im clearly not working.My father George Fraser passed on his immeasurable skill at finishing a project just before the deadline,an

9、d my mother Elizabeth Fraser showed me how to explain concepts and inform people through years of teaching.This book would also not be possible without Tarek Ziads work on the 1st edition and the solid base on which to build,as well as the work of Development Editor Alex Patterson,Reviewer William K

10、ahn-Greene,and Project Editor Rianna Rodrigues.We might stand on the shoulders of giants,but we also build on a huge structure made by every one of us,so I would also like to thank everyone Ive worked with over the years in all my different teams.Yes,even you.Tarek Ziad is a Software Engineer,locate

11、d in Burgundy,France.He works at Elastic,building tools for developers.Before Elastic,he worked at Mozilla for 10 years,and he founded a French Python user group,called AFPy.Tarek has also written several articles about Python for various magazines,and a few books in French and English.I would like

12、to thank Freya,Suki,Milo,and Amina for being so supportive of all my book projects.About the reviewerWill Kahn-Greene has been building things in Python and other languages since the late 90s.He manages the crash ingestion pipeline at Mozilla and maintains several Python libraries.When hes waiting f

13、or CI to run tests,he builds things with wood,tends his tomato plants,and cooks for four.v Table of ContentsPreface xiChapter 1:Understanding Microservices 1The origins of service-oriented architecture 2The monolithic approach 3The microservice approach 8Microservice benefits 10Separation of concern

14、s 10Smaller projects 10Scaling and deployment 11Pitfalls of microservices 12Illogical splitting 12More network interactions 13Data storing and sharing 13Compatibility issues 14Testing 14Implementing microservices with Python 15How web services work 16The WSGI standard 16Workers,threads,and synchroni

15、city 17A worker pool approach 17Being asynchronous 18Twisted,Tornado,Greenlets,and Gevent 19Asynchronous Python 19Language performance 21Summary 24Table of Contents vi Chapter 2:Discovering Quart 25Making sure we have Python 27How Quart handles requests 28Routing 32Variables and converters 33The url

16、_for function 37Request 37Response 38Quarts built-in features 40The session object 40Globals 41Signals 42Extensions and middleware 44Templates 46Configuration 48Blueprints 50Error handling 52Custom error handler 52A microservice skeleton 54Summary 57Chapter 3:Coding,Testing,and Documentation:the Vir

17、tuous Cycle 59Different kinds of tests 62Unit tests 63Functional tests 67Integration tests 70Load tests 71End-to-end tests 75Using pytest and tox 76Developer documentation 80Tools for documentation 81Version control 85Continuous Integration and Continuous Deployment 86GitHub Actions 87Documentation

18、88Coveralls 89Badges 89Summary 90Table of Contents vii Chapter 4:Designing Jeeves 91The Jeeves bot 92User stories 92Monolithic design 93Model 94View and template 97A human-readable view 100Slack workspaces 103Taking actions 107OAuth tokens 108Authentication and authorization 109Background tasks 113P

19、utting together the monolithic design 116Summary 118Chapter 5:Splitting the Monolith 119Identifying potential microservices 119Code complexity and maintenance 121Metrics and Monitoring 122Logging 128Splitting a Monolith 131Feature Flags 135Refactoring Jeeves 137Workflow 139Summary 140Chapter 6:Inter

20、acting with Other Services 141Calling other web resources 142Finding out where to go 145Environment variables 145Service discovery 145Transferring data 146HTTP cache headers 147GZIP compression 149Protocol Buffers 152MessagePack 153Putting it together 155Asynchronous messages 155Message queue reliab

21、ility 156Basic queues 156Table of Contents viii Topic exchanges and queues 157Publish/subscribe 161Putting it together 162Testing 162Using OpenAPI 164Summary 165Chapter 7:Securing Your Services 167The OAuth2 protocol 168X.509 certificate-based authentication 170Token-based authentication 173The JWT

22、standard 174PyJWT 176Using a certificate with JWT 177The TokenDealer microservice 179The OAuth implementation 180Using TokenDealer 184Securing your code 186Limiting your application scope 187Untrusted incoming data 189Redirecting and trusting queries 192Sanitizing input data 193Using Bandit linter 1

23、94Dependencies 195Web application firewall 195OpenResty:Lua and nginx 196Rate and concurrency limiting 199Other OpenResty features 201Summary 202Chapter 8:Making a Dashboard 203Building a ReactJS dashboard 205The JSX syntax 205React components 206Pre-processing JSX 211ReactJS and Quart 212Cross-orig

24、in resource sharing 213Authentication and authorization 215A note about Micro Frontends 216Getting the Slack token 218JavaScript authentication 220Summary 221Table of Contents ix Chapter 9:Packaging and Running Python 223The packaging toolchain 224A few definitions 225Packaging 226The setup.py file

25、226The requirements.txt file 231The MANIFEST.in file 234Versioning 235Releasing 238Distributing 239Running all microservices 242Process management 243Summary 245Chapter 10:Deploying on AWS 247What is Docker?248Introduction to Docker 251Running Quart in Docker 255Docker-based deployments 257Terraform

26、 259Service discovery 261Docker Compose 263Docker Swarm 265Kubernetes 267Summary 272Chapter 11:Whats Next?275Automation 275Scaling 276Content Delivery Networks 277Multi-cloud deployments 277Lambda Functions 277Expanding monitoring 278Making promises 278Summary 280Other Books You May Enjoy 283Index 2

27、87 xi PrefacePython Microservices Development introduces the design and creation of a microservice-based application using the popular Python programming language,and the Quart web framework.In this book,you will learn about microservice architecture and how it differs from the traditional monolithi

28、c approach,what the benefits are,as well as potential trouble that needs overcoming.Who this book is forThis book is for people familiar with the fundamentals of the Python programming language who want to start writing web services,or who have inherited a web service they want to modernize.It is ex

29、pected that the reader will be familiar with simple Python structures such as functions and loops,as well as some more advanced features such as decorators.Some familiarity with using web applications would be helpful,although the fundamentals of application design are covered.What this book coversC

30、hapter 1,Understanding Microservices,introduces the concepts behind microservices,the differences between monolithic applications and microservices,common benefits and pitfalls,as well as testing and scaling.Chapter 2,Discovering Quart,covers the Quart web framework and the ways in which it can resp

31、ond to requests,create templated documents,act as middleware,handle errors,and read configuration.Preface xii Chapter 3,Coding,Testing,and Documentation:the Virtuous Cycle,teaches you about the different types of testing that are possible,what benefits each type has,and how to set up automatic testi

32、ng,as well as generating documentation in CI pipelines.Chapter 4,Designing Jeeves,looks at Jeeves,which is the sample application we use in this book to explain the various concepts behind microservices.We introduce what we need Jeeves to do and describe a monolithic approach to application design,c

33、overing the web API interface,database use,and worker pools.Chapter 5,Splitting the Monolith,builds on the monolithic Jeeves described in previous chapter.This chapter offers guidance on how to identify components that may be good microservices,measuring the effects of changes on the software architecture,and how to cleanly migrate features to new microservices.Chapter 6,Interacting with Other Ser