#48 Garbage collection and memory management in Python

Python Bytes - A podcast by Michael Kennedy and Brian Okken - Lunedì

Categorie:

Tecnica

Sponsored by DigitalOcean. They just launched Spaces, get started today with a free 2 month trial of Spaces by going to do.co/python

Brian #1: The Python Graph Gallery

  • “cool graphs” x “head explodes with options”

Michael #2: pynesis

  • High level python library for using kinesis streams

  • What are Kinesis streams? AWS Kinesis streams

    • Enables you to build custom applications that process or analyze streaming data for specialized needs.
    • Continuously capture and store terabytes of data per hour from hundreds of thousands of sources such as website clickstreams, financial transactions, social media feeds, IT logs, and location-tracking events.

  • High level kinesis client. Support python 2.7 and 3.6, and has helpers for using it within Django.

  • Some features:
    • Supports python 2 & 3
    • Django helpers included
    • Automatically detects shard count changes
    • Checkpoints/sequences persistence can be customized
    • Provided Checkpointer implementations for memory, django model and redis
    • Provided Dummy kinesis implementation for development/testing

Brian #3: Things you need to know about garbage collection in Python

Michael #4: WSGI Is Not Enough Anymore, part 1 and part 2

  • Explores the factors that make WSGI a less attractive option for developing web applications with Python.
  • Most major web frameworks use WSGI (Pyramid, Flask, Django, Bottle, etc.)
  • This has been a major benefit / breakthrough
  • The Web Server Gateway Interface (WSGI) is a specification which was first developed in 2003, and then revised in 2010, in order to create a standard for Python web frameworks to interact with web servers.
  • The bad news is that WSGI comes with two major drawbacks:
    • WSGI compatible servers are synchronous
    • WSGI compatible servers only supports the HTTP protocol
  • Problem 1: Concurrency
    • By design, a WSGI server is synchronous. This means it blocks each request until a response arrives from the application.
    • Scaling is done necessarily via threads (with GIL limitations), web gardens (multiple processes per server), and web farms (multiple servers)
  • Problem 2: HTTP is the only protocol
    • HTML5 introduced, among other things, web sockets, which create a bi-directional communication layer between servers and clients.
    • But they are not supported, so polling (yuck) is the only option
    • Python frameworks which rely on WSGI do not implement web socket communication and must rely on 3rd party solutions and extra components and resources.
  • Part 2 discusses solutions via event driven programming
  • Part 3 (pending) talks about libraries for solving the concurrent problem in Python

Brian #5: Queues in Python

  • Dan Bader
  • I was in search of a LIFO queue and ran across this article by Dan.
  • For LIFO:
    ### collections.deque as LIFO queue
    q = collections.deque()

    # insert elements
    q.appendleft(item)

    #iterate
    for item in q:
        print(item)

    ### queue.LifoQueue
    q = queue.LifoQueue()

    # insert elements
    q.put(item)

    #iterate
    while not q.empty():
        item = q.get()
        print(item)

    ### list as LIFO queue
    q = []

    # insert elements
    q.append(item)

    #iterate
    for item in q[::-1]:
        print(item)

Michael #6: Using Reflection: A Podcast About Humans Engineering

  • by Mark Weiss
  • Check out Jesse Davis’s episode for a starter.
  • Engineering journey, what they value about engineering and skills they have come to recognize in themselves.
  • Dig into what makes teams successful, and how we help them succeed.

Our news

Visit the podcast's native language site