I'm Al Sweigart, author of several free programming books. My latest book is on recursion and recursive algorithms. AMA!

This is the official recursion joke thread. Please keep recursion jokes under this thread.

For all those who don't understand: Recursion is when you solve something recursively.

But what is solving something recursively?

Why don’t they teach kids to write in recursive anymore?

Thanks, Obama.

EDIT: Actually, this joke makes me feel old.

Hi there. I feel very out of my league here and I hope I'm not wasting anyone's time. I have your book and have been working through some projects and have been using Mark Lutz's material for about the past six months with consistency.

I'm coming from a linguistic background and perhaps this is part of my issue, but would you have any wisdom regarding the creative process? I feel like I struggle trying to come up with my own practical applications for what I'm learning and just end up trying to copy the code verbatim that I see. Is that normal or is there another way to approach things?

Edit: How would you go about this issue differently?

This is pretty common. It's easy to learn the syntax to a programming language and the basic concepts, but then you feel like you don't know how to begin creating an actual program. Really, copying code that you've seen is he best way to get experience. I wrote The Big Book of Small Python Projects to provide tons of examples of small running programs. People say you should read the source code of open source projects, but those projects could be under-documented or use who knows what coding techniques or idioms, and that can be frustrating to a beginner. At the same time, I wanted something that was more than just code snippets; I wanted examples that were complete programs that you could run.

There's a thing called tutorial hell or the tutorial desert, where you've done Hello, World tutorials but are still a beginner and aren't ready to write your programs yet. So you start looking up other tutorials, only to find more Hello, World stuff that you already know. It's pretty hard to find learning materials for the intermediate level. So I wrote Big Book and also Beyond the Basic Stuff with Python to try to address readers at that level.

But basically, yes, it is totally normal to constantly be looking up stuff on Stack Overflow and copying other people's code. This is how you build experience.

Your book automate the boring stuff with python gave me the confidence that even I can code, after reading it, I started applying for qa automation and development roles, so thank you so much for writing that ❤️. My question is have u ever felt imposter syndrome, if yes how do you deal with it?

I used to feel like an imposter. And I still do. But I used to, too. But with the popularity of Automate the Boring Stuff, people keep telling me I'm a great programmer (I'm okay; writing books and coding are two different skills). So I'm pretty sure I'm going to get away with it for the rest of my life.

But don't worry. Imposter syndrome is something every developer has, and it goes away after 30 or 40 years.

I'm 'self*-taught,' so I'm missing some of the formal CS background knowledge. If a recursive approach works, are there generally any dangers to using it?

*some mix of py quick, stack exchange, you, and other random resources

The problem of recursion is that it could be difficult to understand and difficult to debug. If your recursive code doesn't do any backtracking, it's most likely much easier to just use a loop instead. By "backtracking", I mean that your code does stuff after the recursive function call. If the last thing in your recursive function is returning the results of the recursive function call, you aren't doing any backtracking and don't need recursion.

Generally, if you have the question, "wouldn't it be easier to code this using a loop?" the answer is 99.9% of time, "Yes."

I’m not Al so feel free to ignore my answer, but generally speaking recursion has much worse performance than a loop doing the same thing. From my understanding, the only time you really want to use a recursive solution is if you absolutely must (as his comment mentions about “back tracking”) or if the performance doesn’t matter and recursion makes the code significantly more understandable.

Both of those cases are rare, so most of the time using recursion is really just showing off.

but generally speaking recursion has much worse performance than a loop

Maybe, maybe not. I thought so too, but I remember testing some recursion-vs-iteration Python example with a profiler and found that the recursive one was faster. For... some reason?

Software is so damn complicated these days that we really don't know how stuff is going to run until we actually measure it. And if we make a slight change we have to measure it again because who knows how that small change affected things.

Why do people preach functional style programming nowadays as opposed to OOP? Especially the usage of recursion.

No clue. I could never really get into it. Immutability is a good feature to have, but you don't need to do functional programming for that. OOP isn't always so great either. Object-Oriented Programming is Bad is a great talk on how OOP techniques are taken too far (or are even a problem when not taken too far).

Considering you write a lot of open source libraries for Python: What's a feature you feel should be added to the Python standard library or an existing feature that warrants significant improvement?

(Thanks for writing AtBS and making it free, by the way!)

I've been working on a module called Humre, which lets you create human readable regular expressions. So instead of this:

>>> import re
>>> re.compile('(?:(?:0x|0X)[0-9a-f]+)|(?:(?:0x|0X)[0-9A-F]+)|(?:[0-9a-f]+)|(?:[0-9A-F]+)')

You can have code like this:

>>> from humre import *
>>> compile(
...     either(
...         noncap_group(noncap_group(either('0x', '0X')), one_or_more(chars('0-9a-f'))),
...         noncap_group(noncap_group(either('0x', '0X')), one_or_more(chars('0-9A-F'))),
...         noncap_group(one_or_more(chars('0-9a-f'))),
...         noncap_group(one_or_more(chars('0-9A-F')))
...     )
... )

Think of it like an advanced form of verbose mode. Humre isn't a new regex engine, it's just a nice wrapper that produces regex strings. It's nice for beginners because it uses real words instead of punctuation marks, but it's also nice for experienced developers because you get all your IDE tooling:

• Your editor's parentheses matching works.
• Your editor's syntax highlight works.
• Your editor's linter and type hints tool picks up typos.
• Your editor's autocomplete works.
• Auto-formatter tools like Black can automatically format your regex code.
• Humre handles raw strings/string escaping for you.
• You can put actual Python comments alongside your Humre code.
• Better error messages for invalid regexes.

It'd be nice if this was added to Python's built-in re module, but that would require a lot of politicking. A lot of experienced devs seem to hate the idea (completely ignoring all the reasons I give for why it's not just for beginners). The main benefit of regex syntax is that it's an established standard, and while adding Humre-style functions to re wouldn't get rid of existing re functions, a lot of people who have already learned regex take a "what's the big deal, just learn regex" approach to this new thing.

I've always thought the reason why my books sell well is because I go to big lengths to make programming understandable to beginners. Authors with years of software dev experience tend to write books that are technically accurate but present an unforgiving steep learning curve. Which is funny, given that Python's popularity comes from it's readability.

Authors with years of software dev experience tend to write books that are technically accurate but present an unforgiving steep learning curve. Which is funny, given that Python's popularity comes from it's readability.

This is very true. I remember the confusion I felt the first time I looked at a list comprehension - this would be considered 'easily readable' by current me. I work with some python programmers who are much smarter and their definition of readable tends to shift with their years of experience. At some point, the readable Python code in the expert's eye is not that much different from the "line noise" they accuse Perl of being (well, maybe a little different).

It is clear you are coming from a pedagogical metric of readability for beginners which is a much more reliable standard than what feels readable in your head - thank you for doing what you're doing and continuously striving to make it a little easier for beginners.

Yeah, I really wish "list comprehensions are just a shortcut way to write a for loop that creates a list" was the way list comprehensions were explained, followed by an example like this:

This list comprehension:

myList = [str(x) for x in range(10) if x % 2 ==0]

...is the exact same thing as this:

myList = []
for x in range(10):
    if x % 2 == 0:
        myList.append(x)

List comprehensions are just shorter to type and fit on one line of code. That's all.

The first and last thing that come to my mind are how am I going to tell a beginner why they need to know a thing. I pretend that the reader doesn't care at all about cool programming techniques or whatever, they just want to learn how to get a thing done.

The first and last thing that come to my mind are how am I going to tell a beginner why they need to know a thing.

Oh gods yes. This is my.... favourite?... rant... No one wants to talk about why. Kinda fundamental if you ask me. Why do you do it This way? (what problem does it solve, trap avoid, situation make easier, code execution speed up)

The other one that gets me is I have a problem to solve/thing I want to do. What pattern or approach fits that use case

Documentation: This is the syntax. These are the options. These are the exceptions. Occasionally, this is what this application does.

and nothing to guide you TO that application as a potential solution.

This is not just a beginner problem. I constantly find programs that solve a problem that I just never encountered before. Long ago learned that if there was a problem that seemed to have no existing solution likely it was using different keywords or for a different problem domain (usually much broader).

Side note on this side note... folks who's response to questions include belittling the asker and implying they are lazy or deficient somehow... well... special place in hell. (I've encountered this not just online but face to face, more than once. grrr)

Yeah, trying to learn some programming thing from reference documentation is like trying to learn Spanish by reading an English-Spanish dictionary.

I’d call it humex. Like regex but it’s human expressions.

Also thanks for your books. I started coding by reading them a couple years ago. Currently working as a programmer and doing a masters in CS.

...

...

That's actually pretty good.

"We make stir fry on Fridays. Want to know what we call it?"

"Stirfriday?"

"No, but that's much better."

I copy my regex from stackoverflow and call it a day 😶

I copy my coworker's code and post it stackoverflow.

This is so cool!

:)

Hey Al! Just got the Automate the boring stuff book. I see you also have videos on Udemy. Would it be best to use them together?>

Yes. Really, I created the Udemy course as a way to promote the book and for learners who prefer video. But the second edition book is more up to date and contains covers more content. It is nice to see someone entering code into an IDE and running it to get a general feel for what programming looks like though.

The first 15 videos of the online course are free to watch on YouTube: https://www.youtube.com/playlist?list=PL0-84-yl1fUnRuXGFe_F7qSH1LEnn9LkW

Im currently in university and am taking algorithm analysis this semester. We will be covering recursive algorithms. What topics or algorithms would you consider as essential to understanding the subject?

Understanding what the call stack is and that a separate set of local variables are created for each function call and not each function. I talk about this in the first chapter.

Also, that there's nothing magic about recursion. Anything you can do with recursion can be done with a loop and a stack. Anything. Here's an iterative version of the infamously recursive Ackermann function. Notice that there's no recursion because there's no recursive function call (there isn't even a function in this program!) But it still displays the same output as the recursive version.

Recursion is useful when the problem involves tree-like structures and backtracking. Examples of this include maze solving, searching a file system (folders can recursively contain other folders), and the Tower of Hanoi algorithm. Otherwise, recursion is probably overengineering the solution.

Also, there's no such thing as top-down recursion and bottom-up recursion. This is a common misnomer and I've never seen anyone point it out: what they mean is top-down dynamic programming (where you use recursion to break a problem into smaller subproblems and use memoization to handle repeated calculations) and bottom-up dynamic programming (where you start from the base case and build up, without using recursion).

If someone insists that "bottom-up recursion" is real, ask to see an example of bottom-up recursive Fibonacci; their solution won't involve any recursive function calls, so how can it be recursion?

Also, factorial and Fibonacci are the most common introductions to recursion, but you'd never want to use recursive factorial or recursive Fibonacci in the real world. Recursive factorial easily leads to stack overflows (unless you use tail recursion, which Python, Java, and most JavaScript engines don't support) and recursive Fibonacci has so many repeat calculations it's impossible to solve quickly unless you use memoization.

It's kind of amazing that I never found people saying these straight forward things in my research into how recursion is taught.

Loved your book! What do you think the future holds for Python and languages in general? Do you see any possible up and coming languages?

These are languages I haven't done a lot of development in, but I think hold a lot of promise:

• Rust - A systems language that produces compiled binaries that takes code security very seriously. I think it's time we admit that after decades of making the same mistakes in C++ over and over, we need a better way to write code.
• Kotlin - This is like Java++ the way C++ was an improvement on C. Kotlin fixes a lot of the problems in Java, but at the same time it produces JVM byte code and is interoperable with existing Java code, so you don't have to rewrite all your existing Java code in Kotlin.
• Dart - I haven't paid attention to this in a long time, but for a while it seemed like this could be the JavaScript killer. I don't think it ever really took off though, which is a pity. I'm hopeful for PyScript to become a way to put Python in the browser.

Python is 30 years old at this point, so I tend to be conservative about adding features that are only a small improvement to the language. I can see how improvements are better, but at the same time we run the risk of breaking "there should be one and preferably only one way to do it". For example, f-strings are great and I love them. But this means we have tons of ways of doing string interpolation: the %s way, the format() method, and now f-strings. It would have been nice to just have f-strings from the start. So unless a new feature that changes Python substantially is as great as f-strings, I'd tend to say we should forego it to keep the language simple.

I'd love a book on Rust from you!

I'm currently reading and enjoying Command-Line Rust by Ken Youens-Clark.

congrats on the book, and on previous books. just wanted to comment that i appreciate the choice of including javascript, and i think a lot of other people might too

i learned python for vfx work and for scientific computing, so i've used it plenty. but since learning javascript and using it in the browser for graphics and signal processing prototyping, the fact that you can write straightforward inner-loop/for-loop code without feeling like the language is on your back for doing something terribly "unpythonic" is awesome (where i interpret "unpythonic" in a lot of cases to actually be something like, Don't go there cause this language is too slow for that!). but then you also have like map/filter/reduce available, do you can use all the modern, ergonomic stuff too

also, with javascript examples available in a straightforward html page, the focus is on what the code *does*. you're looking at the output, and see that 1) it works, and 2) it does something of interest to you, before you view source and check out what the interesting thing is made of. i feel like a lot of people who aren't really from a programming background might feel the same: that they don't really want to "just show me the code" from the outset; they want to see from the outset that what is being produced is worthwhile enough to justify the investment in trying to understand what's going on. the focus on "the rendered output of the program" rather than on "pages full of text that i have to figure out how to run just to see what it does" is helpful for that kind of audience

(for reference, the types of things that got me interested enough to learn javascript were like, the big page of D3 examples, P5.js and creative coding with Processing, looking at js1k/demoscene kind of things. all stuff that can be saved out to an html page, explored with the browser console, and put in a codepen or jsfiddle)

anyway, i saw in another answer that you mentioned a rosetta code style book, and i thought that would be really good too, cause a lot of people learn well that way too

Yeah, almost every programming language is similar enough that I could do a one-to-one translation between Python and JavaScript if I kept the programs simple. They all have loops and functions and variables.

I also wanted to have runnable programs in this book, and not just "code snippets". Like, you should be able to copy the code into your computer and run it to see what it does. It shouldn't just be an example the reader looks at and is left to "figure out" what it does; the reader might not be able to or might think it does some it doesn't. I've never liked these examples. They're always missing an import statement or require some setup steps they don't specify.

Hi.

My main thing is more traditional OOP, but I'm kind of learning Python right now.

I was wondering what your thoughts are on Python's strongest aspects and best uses?

Some people joke that Python is the second best language for every problem. The kernel of truth here is that Python really is a good all-around language. It's used in web apps, machine learning, data science, devops, etc. It's easier to list the areas where Python isn't widely used:

• Embedded programming (though Micropython addresses this)
• Gaming (though if you're making 2D indie games, Pygame is great. And Godot has GDScript, which is based on Python syntax)
• Mobile apps (though BeeWare is addressing this)

What's great about Python is that it doesn't force you to any paradigm. If you want to use OOP, you can. If you want to write Python code in a functional style, you can. If you want static typing (or if you want dynamic typing with the option of gradually switching to static typing), you can do that too. But the language doesn't force you to use any of these.

On a related note, why do you prefer Python? Or what made you write your book on Python as opposed to any other language?

Python's a great general programming language with readable syntax. I feel like a lot of computer nerds like that programming is difficult. Like, maybe that makes their knowledge all the more impressive because it's exclusive?

But I say that computers were made to make human lives easier, and not the other way around. I remember when Automate the Boring Stuff with Python first came out, I posted about it on Hacker News and one person commented, "This is cool, but couldn't you do all of this with shell scripts?"

The answer is, probably yeah. But it would be hard, and tedious. And inaccessible to beginners. But if you're a software developer with a decade of experience, yeah, you could do all this stuff with your hard earned knowledge. I just wanted to make it possible for beginners to do it without having to spend ten years programming.

In the movie Lord of War, Nicholas Cage plays an arms dealer and he's talking to this other arms dealer, who tells him, "I don't just sell weapons: I pick sides."

The Python community is great. PyCon was the first and still is the best tech conference I've ever been to. The people are welcoming, and you just don't get that same alpha-nerd vibe that other tech communities can have. So I really appreciate Python in multiple ways.

Thanks for the great answer! It makes me wonder, if I’m not too late to ask, what are your thoughts on Ruby?

I personally love Ruby for similar reasons - I think it has really readable syntax and great documentation. After getting used to it, I find my brain wishing other languages worked the same way, especially because it makes logic and anonymous functions really easy.

I promise I’m not trying to start some flame war, I just really enjoy writing code in Ruby but I’ve found it isn’t as popular, so I’m interested in hearing educated opinions on it!

I like Ruby, but love Python. But also, I'm pretty sure Ruby's star has faded where we are now in 2022. I look at the top 100 books in various software-related categories on Amazon, and Ruby just isn't there. TIOBE's index (flawed as it may be) has Ruby at #20, after Perl and R. Really, the things that Ruby is good at are also the things that Python is good at. By all means, learning more programming languages is a good idea. But I think Ruby's initial success was due to Rails, which has also faded in popularity. Career-wise, Python (or even PHP) would be a better move.

(And I don't think monkey patching was ever a good idea, in general.)

This is awesome I struggle with this concept a lot as I have only been learning to program a few months.

What makes recursion useful, and where is it under-utilized by self identified beginners/intermediate programmers ?

Recursion is especially useful when you have a problem that involves tree-like data structures and backtracking. So maze solving is an example: the intersection in the maze is a node in the tree graph, and the branching paths from that intersection to the next intersection are the branches of the graph to other nodes. A dead end in the maze is a node without any branches, at which point you backtrack to an earlier node and take another path. If you've already taken all paths, you backtrack again until you find an intersection whose branches you haven't exhausted yet.

The thing is though, if you don't need this backtracking behavior, most likely you don't need recursion and using a recursive function instead of a loop is just making your code harder to understand.

Why/when did you decide to publish online for free? Btw, I've found your books to be very valuable. Thank you!

When I first started writing books, I worked as a software engineer and the books were just a part time hobby. I found out about the Creative Commons license, so I published the books under that.

But when I started to take the commercial aspect more seriously, I found that a CC license was a huge benefit: people could read it for free online and generate word of mouth.

And anyway, I need intellectual property protection mostly to stop other people from taking my books and selling them, rather than to stop readers from downloading them. That's the kind of thing that costs me actual sales. Readers downloading pirate copies is as harmless as libraries loaning out copies of my books.

My god, you're a gem.

This one time the guy in front of me at the ATM was having trouble and he left without realizing it dispense $20. I called him back over to grab it, and he said, "Dude, you're the most honest person I know."

I thought about it, and on the one hand, yeah sure. But also at the time I was in a pretty safe place finance-wise. I didn't steal his $20 because I didn't have to. I was still going to eat that day if I didn't take the $20.

It's easy to do the "right" thing when you're in a safe place. And it's been this way for most of my life: I lived on rice and ramen, but my parents were able to pay for most of my college tuition (at a tax-funded state university, UT Austin) and living expenses. I graduated without debt. That made it easy to take chance on moving to San Francisco, which led to a ton more opportunities. Risking failure meant risking embarrassment, not risking homelessness.

Whenever you see magazine articles that ask "how did this 23 year old accomplish so much at such a young age?" the answer is always "rich parents". Bill Gates had a family connection. Jeff Bezos' parents had a spare quarter million dollars to invest in Amazon in 1995. Elon Musk's father owned an emerald mine in Africa.

It's a nice boost to my ego, but I try not to let the compliments make me forget where I came from and who has helped me along the way.

Hey Al-

First, I've seen how often you post free access to your course. I just wanted to say that that seems mighty decent of you. Thanks for doing it!

My question: there's been a ton of new and cool features added in the last few versions of Python. Things like the walrus operator and lru_cache are personal favorites. Do you think that there's anything relatively recent that's flown under that radar that deserves more attention?

I feel like everyone knows about f-strings and type hints. But 3.10 introduced a library for TOML. At first, I wondered what the big deal was: first we had XML, then JSON, then YAML... how many data formats do we need? But when I read up on TOML, I realized it's a format that especially suited for humans writing configuration files in text editors (rather than a format for machine-produced data). TOML syntax is similar to .ini files. It's actually nice, so if your program has config files that are meant to be editable by a person with Notepad, then consider TOML. I wrote a blog post about it.

Heh, I still need to read up on 3.10's structural pattern matching. But I heard "it's more than just a switch statement" so I'm interested in it.

You're definitely right about f-strings and typing! I feel like those have become fairly indispensable already. That's interesting about TOML, as well, I'm going to have to read up some more on it, thanks!

As an aside: it took me a second staring at it to get "Mordnilap" in your post, but it gave me a chuckle.

"Palindrome Mordnilap" was a Dungeons & Dragons character I created. I think alcohol was involved.

Aware of your book from programming podcasts, but I read your name here as "Ai Sweigart" i.e. artificial intelligence Sweigart.

Are you an Ai, Mr. Sweigart? Is your net worth truly $127.3 million? Do you play a lot of basketball given your height of 6'8"?

(Jokes aside, looking forward to reading your book!)

Yeah, I think around 2015 is when AI became enough of a household name and everyone started reading "Al" as "AI". It was a problem before then. :)

For those who don't know, I have a bunch of fake facts about myself at the bottom of alsweigart.com to pollute data mining bots.

This is not /r/NominativeDeterminism?

Sorry, wrong sub.

Hiro is my middle name. :)

Where do you commonly see people using recursion when a simple loop would suffice?

Also, thanks for all of your contributions to helping everyone learn. As a CS teacher, I aspire to be as clear in my explanations.

The first thing that comes to mind is anything that involves tail recursion or tail call optimization. I have the opinion that TCO should never be used, and it's a sign that you should use a loop instead. TCO can only be done when your recursive function doesn't need the call stack. So if any recursive algorithm can be done with a loop and a stack, and you don't need the stack, then just write the code as a loop. TCO often requires you to mangle your code, and anyway, Python, Java, and JavaScript don't implement TCO anyway.

I feel like the best teaching examples for recursion are Tower of Hanoi, maze solving, searching a file system, and (if you have a turtle library) drawing trees and other fractals.

The factorial and Fibonacci examples that are commonly used as terrible. Factorial can easily cause a stack overflow (unless you use TCO) and Fibonacci has repeat calculations that require memoization. Instead, I'd use a "count down and up" example as my first recursive function, as well as a function that does nothing but cause a stack overflow. I describe them in Chapter 1.

The times I use tail recursive function is when I don't know how many items I'm iterating over, with an API being the prime example. I just write a function get the api data and then if "next page" token exists in the data, call the function again, is this a bad thing?

I'm not sure how to write a loop for this and the recursive way seems super clean and readable to me.

If your code uses tail recursion, it looks something like this:

def myFunc(page, token):
    # do stuff
    if token.has_next_page():
        # recursive case
        myFunc(token.next_page(), token)

The last thing the function does is recursively call itself. If it does that enough without returning, it causes a stack overflow. To borrow a baseball term, that's kind of an "unforced error" because it doesn't need to be a recursive function in the first place. You could write it as:

while token.has_next_page():
    page = token.next_page()
    # do stuff

(Though, of course, this depends on what your code looks like. I'm just taking a stab at it.)

After all, everything you can do with recursion you can do with a loop and a stack. And for tail recursion, where the last thing the function does is call itself, you don't need the stack. So just use a loop.

10 years ago I got consistently asked in interviews Write a function that returns the nth number of Fibonacci sequence.

I always answered using a loop and array and filling in the array with the sum of the previous two values. However every interviewer told me “is there a better way?” And implied that they wanted the answer using recursion. Back when I didn’t know the recursion answer I failed those interviews but I went to study that method later so I know it now. Its a lot less intuitive. But still…

Is recursion really superior to the loop array method?

Speaking as someone who has now written an entire book on recursion, ABSOLUTELY NOT*.

*The times that recursion is great is when you have a problem that involves a tree-like data structure and backtracking. If your problem doesn't have that, then you shouldn't recursion.

There's nothing magical or automatically "more elegant" about recursion. Anything you can do with recursion, you can do with a loop and a stack. And if you aren't doing backtracking, you don't need a stack. So just write code with a loop.

In fact, the two common introductory examples of recursion, factorial and Fibonacci, are things you'd never want to use recursion for in real life. Factorial suffers from stack overflows for large enough numbers (unless you use tail recursion, which Python, Java, and JavaScript don't implement), and Fibonacci very quickly slows down due to repeat calculations (unless you use memoization, but at that point you're adding advanced techniques to make up for the shortfalls of the advanced technique you are applying to an otherwise simple algorithm).

If you ever find yourself asking, "Wouldn't this be easier with a loop?" the answer is "Yes." 99.9% of the time.

Programmers are weird. Sometimes we like programming more than just using computers to solve problems.

Big fan of your books approach skipping theory. I learned theory afterwards and it was so much easier.

How mature do you think BeeWare is currently?

What's your opinion on PyScript?

For those that don't know, BeeWare is a Python framework for creating apps for Android, iOS, Mac, Linux, and Windows in Python (kind of like Electron). I haven't worked with it much, but I am excited about it. The main person behind it, Russell Keith-Magee does a lot of great organizing work behind it, and I see it as the best chance for Python on mobile devices. He's recently been hired by Anaconda to work on it full time, so if anything, BeeWare has even more promise than ever.

Like everyone else at PyCon this year, I was blown away by the keynote about PyScript. But I also realize that it's still in alpha, so it's not going to replace JavaScript anytime soon (if ever). Still, having Python in the browser represents one of the few remaining lands unconquered by Python (along with embedded programming and AAA gaming.)

I haven't had time to dig into either too deeply though. They're on the back burner though.

Python famously has it's built-in recursion depth limit (admittedly for good reason). Usually answers to dealing with it range from such broad ideas that sidestep the issue as "don't exceed it, duh" to "just change the limit" to "do it in C". Do you think anything more elegant exists?

Not really. I think there's a good case to be made that if your recursive algorithm truly does need to go more than 1,000 function calls deep, then you should probably write it as a loop and a stack (rather than use the call stack as your stack).

For example, flood fill can easily break this limit with large enough images. But flood fill doesn't have to be solved recursively. You don't even need a stack; you could use a set data structure instead to keep track of the neighboring pixels you need to check.

I'd recommend against changing the limit with sys.setrecursionlimit(). On my Windows machine it tends to crash around 1,800 to 2,500 function calls anyway.

ACTUALLY, I just checked with Python 3.11 which apparently has changed the way the call stack works. I'm able to set the recursion limit to something like 999,999 and it seems to work fine. Still, this is only for Python 3.11. But maybe this does mean you can make deeply recursive functions in Python.

What's a good example you can think of where recursion is the best tool for the job?

Anytime you need backtracking:

• Maze solving
• Searching file systems
• Tower of Hanoi (this one is actually really good for recursion, because the iterative solution is huge pain to write)

All of these do backtracking, which requires a stack. If your problem doesn't require stack, then it can be solved with just a loop. So just use a loop and keep your code simple.

First off, I have to say thank you for writing Automate the Boring Stuff with Python. I read the book and did your Udemy course a little over two years and it inspired me to go back to school for computer science. I am still using things I learned from that book today. In fact, Just a few hours ago, I used the pyautogui in a script a script to help my Grandpa who suffers from Parkinson's disease sign into his email.

Anyways, do you plan on doing a Udemy course to go along with this book as well? I found the combo of a book + video tutorial to be really awesome when going through Automate the Boring Stuff with Python.

That's awesome to hear. :)

Yes, I plan on finishing the Udemy course for Beyond the Basic Stuff with Python and then make a Udemy course for the recursion book as well. But I have some other book projects I need to finish up first. It... might be a while.

For someone coming into the development world, from the Ops side (sec, sysadmin, etc) with no degree, what would be your top recommendation?

I mainly have a bash/PS background, and am slowly but surely learning Python. I'm transitioning into the DevOps world.

Definitely Python. I'd say Python has filled all the sysadmin/devops niches that Perl used to have. And at a certain point, you really do need an actual programming language rather than just making bigger and bigger shell scripts.

Definitely am learning Python. I've created some basic apps to automate a few tasks. Any recommendations within the Python world?

Top recommendations as far as books? Aside from mine, Python Crash Course is pretty good. Python Distilled, The Python Pocket Reference, and The Quick Python Book are good if you can program in another language already. Once you have the basics down, Effective Python and Fluent Python are great.

Checking out PyCon talks on https://pyvideo.org is a good idea as well.

Hello ai Sweigart, glad to see someone built you a humanoid form, do you wish they gave you more hair?

Life stress would just cause me to pull it out again.

Does your book cover un-recursing functions? I find recursion to be extremely natural feeling, but trying to write a depth first search without recursion is absolutely mystifying to me.

Yes. Chapter 2 compares recursion and iteration and has examples of converting one to the other. I also have an iterative example of the notoriously recursive Ackermann function to prove that any recursive function can be made iterative.

Any recommendation for learning python beside your book and bootcamp?

Python Crash Course is great. If you already know programming but want to learn Python, The Quick Python Book, Python Distilled, and The Python Pocket Reference are nice.

Thanks for doing this! I am a total beginner to the point I feel like I don’t belong here asking a question, but since I’m reading your book and taking the class, I didn’t want to miss the opportunity.

I have no reason to think that anything is “missing” from the book or class, but I feel like in any course for beginners, there’s something that when the course is done, beginners may find they still don’t know. Is there anything I should look out for like that? If not, what would you recommend to a beginner (me) who seems to understand the concepts when learning, but has trouble imagining what types of programs to create with that knowledge?

I feel like I don’t belong here

You're in good company; everyone feels like that.

...but has trouble imagining what types of programs to create with that knowledge?

This is also pretty common. Tutorials will teach the language syntax, but you're often left on your own when it comes to creating programs. Really, the best way through this is to start looking at example programs. I'm not sure if there's a better way through that. Open source software can often be complicated and not well documented, so I wrote a book of short and simple programs for intermediate readers to check out to see how all these programming concepts they know are used in real programs.

I got better at programming this same way: you get enough experience with code and pretty soon the future problems you have to solve are like the ones you've seen in the past.

Do you see Python ever getting any significant performance improvements? As much as I love it, even its functionality as a “glue” between highly optimized C/fortran code can be noticeable. And since type hints are already becoming very popular, maybe they could enable some optional performance improvements by automatically jitting functions or something like that?

This is above my pay grade, but I don't think it's out of the realm of possibility. I believe most of Python's "slowness" comes from its dynamic typing. (Though "Python is slow" is sort of a misunderstanding, but that's a whole other rant.) Python did get performance improvements to dictionaries in 3.6 I believe, and 3.11 also has performance improvements too. So there's room for improvement, even though Python might not be as fast as a compiled systems language.

I just wanted to let you know I appreciate you providing tons of no-cost options to learn to code. I never learned well in a classroom but had always been comfortable with technology and your examples and approach to teaching the concepts in "Automate the Boring Stuff" broke down walls that I had and I would consider myself dangerous with python thanks to you!

​

Question:

After completing and applying what I have learned in automate the boring stuff, do you think it is more important to learn underlying computer science or would it be more beneficial to learn different algorithms next?

I think after learning the basics, going into "data structures and algorithms" (those are terms you want to google) would be a good idea. This is the basic freshmen course for CS majors, and probably like 20% of the usefulness of my degree. There's a good free course from Coursera call the Algorithmic Toolbox.

I'd also learn regular expressions, recursion, source control (that is, Git and GitHub), and get comfortable using a debugger and logging system (like Python's logging module). And also learn Big O algorithm analysis. I cover that in my free book Beyond the Basic Stuff with Python but Ned Batchelder did a great job covering it in a half hour PyCon talk.

u/AlSweigart can you use python to automate a phone dialer through kitty, on offshoot of putty?

Probably. This is something I haven't had time to look into, but Paramiko is something that comes up when I look into SSH with Python. You could also always use PyAutoGUI to control the mouse and keyboard to interact with kitty, though this would be a real brittle way to go about it. That is, your program could easily get out of sync with kitty with it's clicks and key presses and stop working.

How well is tail recursion supported in python? What is your approach to recurse over 1000 times in python (passed the 'maximum recursion depth exceeded' error message)

Tail recursion isn't supported in CPython (the Python interpreter you download from python.org) and never will. Guido van Rossum explains why in a couple blog posts (first, second) Python won't ever have tail call optimization (TCO). Generally, if your recursive algorithm is making function calls 1,000 levels deep, you should probably use a loop and a stack.

However, I discovered today that Python 3.11 can support basically unlimited stack sizes with sys.setrecursionlmit() (previously it was limited to about 1,700 to 2,500 calls deep, from my testing.) But remember, your user has to use 3.11, otherwise their program will crash.

How would you improve the Jupyter-style coding for data processing in Python? In terms of heuristics and maybe redesigning the format. Its a very approachable way to code but at the same time it can get very messy very fast.

I don't have enough experience with Jupyter notebooks to say. Notebooks are a great way to get a lot of people up and running with Python code (and sharing code with others), but it does have some awkwardness compared to just running code from a terminal.

What CAN'T python do? Or at least do well, in your opinion?

The three areas that Python hasn't conquered yet are:

• Embedded (but Micropython can be used here)
• AAA Gaming (but Pygame and Godot's GDScript can create some good stuff)
• Mobile apps (but BeeWare is promising here)
• Front-end web apps (but PyScript holds promise here in the future)

When is automate boring stuff with python on udemy is going to be on sale?

The dark secret of Udemy is that they're like department stores and they always have a sale going on. Never pay full price for an online course. Open the website in your browser's privacy mode, and their website will think you're a new customer and offer you the discounted price.

Otherwise, I give out 2,000 free sign ups at the start of each month on the r/learnprogramming sub. But you can also watch the first 15 videos on YouTube right now for free.

Hey Thanks Al! Always recommend your book first for starting python/programming. How do you continue learning new things in python? There seems to be an intimidating step to overcome when advancing to a point in python (or maybe just not exposed to much given my work area).

I always have a stack of books to read. Fluent Python and Effective Python are great. The CPython Internals book by Anthony Shaw is on my list. Code: The Secret Language of Computer Hardware and Software is something I need to finish reading even though I already learned all that stuff in college but it'll give me ideas for presenting those concepts (the NAND To Tetris book is also similar).

And I have a ton of PyCon talks I've been meaning to watch and take notes. Those are all up at https://pyvideo.org

And then to make sure I've learned the topics I've read, I try to write blog posts that force me to explain the topic to others (and find any holes in my own understanding).

What is the future of capitalism?

"Bourgeois society stands at the crossroads, either transition to socialism or regression into barbarism."

But I'd be happy if capitalism just stopped making video games suck. The quarter-sucking game design of 80s arcades infected home consoles for a long time. Now we have microtransactions and always-online requirements and loot boxes and UI dark patterns that basically turn video games into Skinner-box casinos for children. It's 10x worse on mobile. We also have level grinding just to pad out the hours for a game. All from an industry that deeply exploits its workers. Burnout and crunch is so damn common in the games industry and has been for decades. EA_spouse (for those old enough to remember) made her post in 2004, for crying out loud.

I want shorter games with worse graphics made by people who are paid more to work less and I'm not kidding.

By chance did you try Noita?

Nope! But it looks interesting. I should try to get through some of my Steam library before I buy new games though.

I remember you said you're going to update the Udemy course to match the newer edition. How's that coming along?

Heh heh... any day now...

Actually, it'll probably be pushed back even further. I'd like to put out the third edition of Automate maybe sometime late 2023, and then update the course for that. But I also have other book and software and online course projects to do before then.

I once jokingly asked a friend, "Would it help my productivity if I developed a coke habit?" and without missing a beat she said, "At first, yeah."

What would you say to me if I tell you that I hate recursion (and I have a very hard time understanding it beyond the absolute basic)?

I'd say you have a typical and completely warranted attitude to it.

But then I'd ask you to read my book to see if it helped you.

man, ur a gem for validating my hatred of recursion. and for making the book free.

follow-up question, how useful is recursion and what is it used for? because afaik it's not used in writing backend, frontend, and prbly not for coding MS Office or Excel.

If you have any algorithm that works on similar subproblems and does backtracking, then recursion is a good idea and will probably be easier to write than an iterative solution.

For example, a maze is just a series of intersections where you try one path, which leads to other intersections (this is the similar part) and then you backtrack to earlier intersections if you reach a dead end.

In more literal terms, if you have a recursive function that looks like this:

def myRecursiveFunction():
    # code that does stuff
    myRecursiveFunction()  # the recursive case
    # code that does other stuff

...then you have a recursive algorithm that backtracks so it can do the "code that does other stuff" part. If the recursive function call is the last thing the recursive function does, that's a sign that you can just use a loop instead.

But...loop? You literally said that. Also, wouldn't it run into max depth stack issue or whatever that's called?

Oh yeah. So, recursion is fine for a filesystem crawler, because even though it's possible, it's highly unlikely that a filesystem will have 1,000 levels of folders within folders. So recursion is fine there and you likely won't hit a stack overflow. But for other problems like factorial or reversing a string, you can easily run into a stack overflow.

You never need to use recursion, but when your problem involves backtracking it can be easier to write the code the recursive way. All other times though, it's probably easier to write it the iterative way.

After python, which programming language do you prefer to use? Which one do you recommend?

Probably JavaScript. Not because I like the language, but because it's so widely used and you have to use it to do web front-end stuff.

But I'm taking a closer look at C# and Java (which have similar syntax). And Kotlin, Rust, and Go are on my list of languages to get to know better.

Hi Al!

Hope you are doing wonderful. I got into programming and got my first job thanks to your "Automate the boring stuff" book. I hope one day I can return the favor by making something awesome as your books that help others.

Do you have some topics you would like to cover in a future book?

I'm currently working on a book of simple programming exercises. I feel there's plenty of "leet code hacker rank" puzzle/challenge sites, but not a collection of really easy programming problems. So I plan on making it a 99 cent ebook.

I have other ideas for programming books:

• Math for programmers (like, basic arithmetic and the kind of math average programmers actually use, not calculus/linear algebra/discrete math specialist stuff)
• A rosetta stone book that shows equivalent code between Python and JavaScript, so that if you know one language you can learn the other.
• A book that teaches Python using video game metaphors. (This idea is kind of meh.)
• A book that teaches programming with Python that doesn't have any text, only pictures. Like a Lego or IKEA instruction manual. You wouldn't even have to know English to read it. I have no clue how I'd pull this off.)
• A book that teaches you how to program using ASCII art animations that I call scroll art.
• A book on all the different types of concurrency: the multiprocessing module, multithreading, asyncio, etc.

With recent pip malware discoveries, would you suggest someone who knows nothing about security to use anaconda and virtualenv as the go-to tools to prevent malware from packages? Any other suggestions?

Nah. PyPI (where pip downloads packages from) has two-factor authentication and is making a push to make sure maintainers of critical packages (and their dependencies) use it. They also take active steps to prevent malware and package-name squatting. And virtualenvs won't prevent malicious code from damaging your computer anyway. I don't see a problem using pip & PyPI that can realistically be prevented by using Anaconda's package index instead.

Can you automate some boring task using recursion?

Maybe? I haven't encountered any. Python has os.walk() so you don't need to write a recursive function to crawl through a filesystem. But there could be cases where recursion would be handy.

But this is why I didn't cover recursion in Automate; it's unlikely that you'll need it when writing little automation scripts.