Learn Python ProgrammingSecond EditionA beginner's guide to learning the fundamentals of Python language to writeefficient, high-quality codeFabrizio Romano
BIRMINGHAM - MUMBAI
Learn Python Programming SecondEditionCopyright 2018 Packt PublishingAll rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted inany form or by any means, without the prior written permission of the publisher, except in the case of briefquotations embedded in critical articles or reviews.Every effort has been made in the preparation of this book to ensure the accuracy of the informationpresented. However, the information contained in this book is sold without warranty, either express orimplied. Neither the author(s), nor Packt Publishing or its dealers and distributors, will be held liable forany damages caused or alleged to have been caused directly or indirectly by this book.Packt Publishing has endeavored to provide trademark information about all of the companies and productsmentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee theaccuracy of this information.Commissioning Editor: Richa TripathiAcquisition Editor: Karan SadawanaContent Development Editor: Rohit SinghTechnical Editor: Romy DiasCopy Editor: Safis EditingProject Coordinator: Vaidehi SawantProofreader: Safis EditingIndexer: Mariammal ChettiyarGraphics: Jason MonteiroProduction Coordinator: Shantanu ZagadeFirst published: December 2015Second edition: June 2018Production reference: 1280618Published by Packt Publishing Ltd.Livery Place35 Livery StreetBirminghamB3 2PB, UK.ISBN 978-1-78899-666-2www.packtpub.com
To my dear dear friend and mentor, Torsten Alexander Lange.Thank you for all the love and support.
mapt.ioMapt is an online digital library that gives you full access to over 5,000 booksand videos, as well as industry leading tools to help you plan your personaldevelopment and advance your career. For more information, please visit ourwebsite.
Why subscribe?Spend less time learning and more time coding with practical eBooks andVideos from over 4,000 industry professionalsImprove your learning with Skill Plans built especially for youGet a free eBook or video every monthMapt is fully searchableCopy and paste, print, and bookmark content
PacktPub.comDid you know that Packt offers eBook versions of every book published, withPDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBookcopy. Get in touch with us at [email protected] for more details.At www.PacktPub.com, you can also read a collection of free technical articles, signup for a range of free newsletters, and receive exclusive discounts and offers onPackt books and eBooks.
ForewordI first got to know Fabrizio when he became our lead developer a few years ago.It was quickly apparent that he was one of those rare people who combinerigorous technical expertise with a genuine care for the people around him and atrue passion to mentor and teach. Whether it was designing a system, pairing towrite code, doing code reviews, or even organizing team card games at lunch,Fab was always thinking not only about the best way to do the job, but alsoabout how to make sure that the entire team had the skills and motivation to dotheir best.You'll meet the same wise and caring guide in this book. Every chapter, everyexample, every explanation has been carefully thought out, driven by a desire toimpart the best and most accurate understanding of the technology, and to do itwith kindness. Fab takes you under his wing to teach you both Python's syntaxand its best practices.I'm also impressed with the scope of this book. Python has grown and evolvedover the years, and it now spans an enormous ecosystem, being used for webdevelopment, routine data handling, and ETL, and increasingly for data science.If you are new to the Python ecosystem, it's often hard to know what to study toachieve your goals. In this book, you will find useful examples exposing you tomany different uses of Python, which will help guide you as you move throughthe breadth that Python offers.I hope you will enjoy learning Python and become a member of our globalcommunity. I'm proud to have been asked to write this, but above all, I'm pleasedthat Fab will be your guide.Naomi CederPython Software Foundation Fellow
About the authorFabrizio Romano was born in Italy in 1975. He holds a master's degree incomputer science engineering from the University of Padova. He is also acertified scrum master, Reiki master and teacher, and a member of CNHC.He moved to London in 2011 to work for companies such as Glasses Direct,TBG/Sprinklr, and student.com. He now works at Sohonet as a PrincipalEngineer/Team Lead.He has given talks on Teaching Python and TDD at two editions of EuroPython,and at Skillsmatter and ProgSCon, in London.I'm grateful to all those who helped me create this book. Special thanks to Dr. Naomi Ceder for writing theforeword to this edition, and to Heinrich Kruger and Julio Trigo for reviewing this volume. To my friendsand family, who love me and support me every day, thank you. And to Petra Lange, for always being solovely to me, thank you.
Packt is searching for authors likeyouIf you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and techprofessionals, just like you, to help them share their insight with the global techcommunity. You can make a general application, apply for a specific hot topicthat we are recruiting an author for, or submit your own idea.
Table of Contents
Title PageCopyright and CreditsLearn Python ProgrammingSecond EditionDedicationPackt UpsellWhy subscribe?PacktPub.comForewordContributorsAbout the authorAbout the reviewersPackt is searching for authors like youPrefaceWho this book is forWhat this book coversTo get the most out of this bookDownload the example code filesConventions usedGet in touchReviews1.A Gentle Introduction to PythonA proper introductionEnter the PythonAbout PythonPortability
CoherenceDeveloper productivityAn extensive librarySoftware qualitySoftware integrationSatisfaction and enjoymentWhat are the drawbacks?Who is using Python today?Setting up the environmentPython 2 versus Python 3Installing PythonSetting up the Python interpreterAbout virtualenvYour first virtual environmentYour friend, the consoleHow you can run a Python programRunning Python scriptsRunning the Python interactive shellRunning Python as a serviceRunning Python as a GUI applicationHow is Python code organized?How do we use modules and packages?Python's execution modelNames and namespacesScopesObjects and classesGuidelines on how to write good code
The Python cultureA note on IDEsSummary
2.Built-in Data TypesEverything is an objectMutable or immutable? That is the questionNumbersIntegersBooleansReal numbersComplex numbersFractions and decimalsImmutable sequencesStrings and bytesEncoding and decoding stringsIndexing and slicing stringsString formattingTuplesMutable sequencesListsByte arraysSet typesMapping types – dictionariesThe collections modulenamedtupledefaultdictChainMapEnumsFinal considerations
Small values cachingHow to choose data structuresAbout indexing and slicingAbout the namesSummary
3.Iterating and Making DecisionsConditional programmingA specialized else – elifThe ternary operatorLoopingThe for loopIterating over a rangeIterating over a sequenceIterators and iterablesIterating over multiple sequencesThe while loopThe break and continue statementsA special else clausePutting all this togetherA prime generatorApplying discountsA quick peek at the itertools moduleInfinite iteratorsIterators terminating on the shortest input sequenceCombinatoric generatorsSummary
4.Functions, the Building Blocks of CodeWhy use functions?Reducing code duplicationSplitting a complex taskHiding implementation detailsImproving readabilityImproving traceabilityScopes and name resolutionThe global and nonlocal statementsInput parametersArgument-passingAssignment to argument names doesn't affect the callerChanging a mutable affects the callerHow to specify input parametersPositional argumentsKeyword arguments and default valuesVariable positional argumentsVariable keyword argumentsKeyword-only argumentsCombining input parametersAdditional unpacking generalizationsAvoid the trap! Mutable defaultsReturn valuesReturning multiple valuesA few useful tipsRecursive functions
Anonymous functionsFunction attributesBuilt-in functionsOne final exampleDocumenting your codeImporting objectsRelative importsSummary
5.Saving Time and MemoryThe map, zip, and filter functionsmapzipfilterComprehensionsNested comprehensionsFiltering a comprehensiondict comprehensionsset comprehensionsGeneratorsGenerator functionsGoing beyond nextThe yield from expressionGenerator expressionsSome performance considerationsDon't overdo comprehensions and generatorsName localizationGeneration behavior in built-insOne last exampleSummary
6.OOP, Decorators, and IteratorsDecoratorsA decorator factoryObject-oriented programming (OOP)The simplest Python classClass and object namespacesAttribute shadowingMe, myself, and I – using the self variableInitializing an instanceOOP is about code reuseInheritance and compositionAccessing a base classMultiple inheritanceMethod resolution orderClass and static methodsStatic methodsClass methodsPrivate methods and name manglingThe property decoratorOperator overloadingPolymorphism – a brief overviewData classesWriting a custom iteratorSummary
7.Files and Data PersistenceWorking with files and directoriesOpening filesUsing a context manager to open a fileReading and writing to a fileReading and writing in binary modeProtecting against overriding an existing fileChecking for file and directory existenceManipulating files and directoriesManipulating pathnamesTemporary files and directoriesDirectory contentFile and directory compressionData interchange formatsWorking with JSONCustom encoding/decoding with JSONIO, streams, and requestsUsing an in-memory streamMaking HTTP requestsPersisting data on diskSerializing data with pickleSaving data with shelveSaving data to a databaseSummary
8.Testing, Profiling, and Dealing with ExceptionsTesting your applicationThe anatomy of a testTesting guidelinesUnit testing Writing a unit testMock objects and patchingAssertionsTesting a CSV generatorBoundaries and granularityTesting the export functionFinal considerationsTest-driven developmentExceptionsProfiling PythonWhen to profile?Summary
9.Cryptography and TokensThe need for cryptographyUseful guidelinesHashlibSecretsRandom numbersToken generationDigest comparisonHMACJSON Web TokensRegistered claimsTime-related claimsAuth-related claimsUsing asymmetric (public-key) algorithmsUseful referencesSummary
10.Concurrent ExecutionConcurrency versus parallelismThreads and processes – an overviewQuick anatomy of a threadKilling threadsContext-switchingThe Global Interpreter LockRace conditions and deadlocksRace conditionsScenario A – race condition not happeningScenario B – race condition happeningLocks to the rescueScenario C – using a lockDeadlocksQuick anatomy of a processProperties of a processMultithreading or multiprocessing?Concurrent execution in PythonStarting a threadStarting a processStopping threads and processesStopping a processSpawning multiple threadsDealing with race conditionsA thread's local dataThread and process communication
Thread communicationSending eventsInter-process communication with queuesThread and process poolsUsing a process to add a timeout to a functionCase examplesExample one – concurrent mergesortSingle-thread mergesortSingle-thread multipart mergesortMultithreaded mergesortMultiprocess mergesortExample two – batch sudoku-solverWhat is Sudoku?Implementing a sudoku-solver in PythonSolving sudoku with multiprocessingExample three – downloading random picturesDownloading random pictures with asyncioSummary
11.Debugging and TroubleshootingDebugging techniquesDebugging with printDebugging with a custom functionInspecting the tracebackUsing the Python debuggerInspecting log filesOther techniquesProfilingAssertionsWhere to find informationTroubleshooting guidelinesUsing console editorsWhere to inspectUsing tests to debugMonitoringSummary
12.GUIs and ScriptsFirst approach – scriptingThe importsParsing argumentsThe business logicSecond approach – a GUI applicationThe importsThe layout logicThe business logicFetching the web pageSaving the imagesAlerting the userHow can we improve the application?Where do we go from here?The turtle modulewxPython, PyQt, and PyGTKThe principle of least astonishmentThreading considerationsSummary
13.Data ScienceIPython and Jupyter NotebookInstalling the required librariesUsing AnacondaStarting a NotebookDealing with dataSetting up the NotebookPreparing the dataCleaning the dataCreating the DataFrameUnpacking the campaign nameUnpacking the user dataCleaning everything upSaving the DataFrame to a fileVisualizing the resultsWhere do we go from here?Summary
14.Web DevelopmentWhat is the web?How does the web work?The Django web frameworkDjango design philosophyThe model layerThe view layerThe template layerThe Dja
Object-oriented programming (OOP) The simplest Python class Class and object namespaces Attribute shadowing Me, myself, and I – using the self variable Initializing an instance OOP is about code reuse Inheritance and composition Accessing a base class Multiple inheritance Method resolution order Class and static methods Static methods Class methods Private methods and name ...