That's not the type-hint you're looking for


#1

In teaching a course this week, we got into a bit of a discussion about type-hinting. At the time, we were looking at a small bit of code involving a class and a pair of functions related to a list of instances:

class Stock:
    def __init__(self, name, shares, price):
        self.name = name
        self.shares = shares
        self.price = price

def read_portfolio(filename):
    import csv
    portfolio = []
    with open(filename) as f:
        rows = csv.reader(f)
        headers = next(rows)
        for row in rows:
            s = Stock(row[0], int(row[1]), float(row[2]))
            portfolio.append(s)
    return portfolio

def print_portfolio(portfolio):
    for s in portfolio:
        print(f'{s.name:>10} {s.shares:>10} {s.price:>10.2f}')

We thought about type-hinting of the two functions and came up with this:

from typing import List

def read_portfolio(filename: str) -> List[Stock]:
    ...

def print_portfolio(portfolio: List[Stock]) -> None:
    ...

At first glance this seems reasonable enough. The read_portfolio() function does indeed return a list of Stock instances and that’s what the print_portfolio() function expects as well. The real problem concerns the filename annotation on the read_portfolio() function. Consider:

 >>> port = read_portfolio('data.csv')
 >>> port = read_portfolio(b'data.csv')
 >>> from pathlib import Path
 >>> port = read_portfolio(Path('data.csv'))
 >>> import os
 >>> fd = os.open('data.csv', os.O_RDONLY)
 >>> fd
 3
 >>> port = read_portfolio(fd)
 >>>

All of these things work just fine. The reason they work fine is that the filename gets passed through to the open() function which recognizes so much more than simple strings.

With this in mind, I’m now wondering if one should annotate the filename argument at all. Perhaps that responsibility for annotating that is someone else’s problem–namely that of the builtin in open() call. Perhaps it would be better to just use a doc-string like this:

def read_portfolio(filename) -> List[Stock]:
    '''
    Reads portfolio data.  filename is passed directly to open()
    '''
    ...

More generally, if a function takes an argument that is simply passed through to another function, should it be annotated at all? Maybe not. I’d be curious to get your thoughts.


#2

So two things. First, I would definitely say that the crazy amount of stuff that open() accepts while not adhering to any specific interface/protocol means there should be a type variable that represents what it can take. And since there isn’t one currently provided I wouldn’t put yourself through the pain of trying to type hint open(); I would say it just isn’t worth the trade-off of trying to get it right versus the type safety you will gain for it. (Although it does open an interesting question of whether you would rather have that function take a typing.io.TextIO to allow people to pass in any file-like object to avoid doing the I/O implicitly.)

Second, I actually would have said print_portfolio() takes an Iterable[Stock] instead of a List[Stock] since the code you shown simply wants to iterate through the portfolio object and you really don’t care what data structure it is, just as long as it implements __iter__() and yields str.


#3

The file example is an example of a more general problem in my opinion. Namely, is it my responsibility to type-hint values that a function merely accepts and then pass onto another function with no-further action. Nothing is done with filename in this example other than giving it to open(). I don’t think it’s my responsibility to type-hint that–chances are I will get wrong anyways.

On the subject of getting it wrong, changing print_portfolio() to use Iterable[Stock] further reinforces my thought that I’m not actually smart enough to write type-hints :wink:


#4

This is why mypy doesn’t have a “require type hints for everything” by default. It’s a lot of work for minimal payoff (typically) when typing everything. If mypy is doing its job appropriately then it shouldn’t be necessary to type hint for stuff “just passing through”. I think this is one of those spots where we as a community have not come up with our own guidelines on how to handle things, so people who are not in an enterprise environment but want to try type hints don’t quite know how far to take any of this.

And I had a few years of production C++ and Java work less than 3 years ago, so this whole “loose typing for arguments, tight typing for return values” burned into my brain from those languages ecosystems. :wink: Basically my rule of thumb is “what do I want this to duck type to?”


#5

It seems like it’d be useful if there were a (succinct) way to reference “whatever type open() accepts” as the type hint for that filename param. [1] It could be present-and-future-proofed so that if open didn’t currently provide a type hint for that parameter, the reference would fall back to Any, but would be forward-compatible once open did provide a type hint.

(I haven’t played much with type hints yet, so I’m not sure how useful this idea is in practice, but it seems pretty useful in theory?)

[1] For that matter, the “filename” parameter would be more precisely named something like “openable”, but that’s probably pedantic and more confusing than helpful. Maybe this is a sign of an overly-overloaded parameter in open's interface?


#6

My interest in all of this is not casual. At the moment, I’m in the process of updating books. Surely, type-hints should be mentioned, but I think there’s a risk of programmers coming from C/Java and thinking that they can just casually slap some type-hints on things and be done with it. However, everything I’ve seen has convinced me that it’s not that simple at all. In fact, there’s a pretty good chance that whatever “hint” is chosen will simply be “wrong” in some way. So, coming up with some kind of practical advice of how to approach the topic seems worthwhile.

My gut instinct is to tell people to NOT put type-hints on code unless they are actually using tools that make use of the hints in some way. I think you’re better off not doing anything at all than trying to make an educated guess of the proper type-hint in absence of tools.