[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Static typing—with annotations—function name & arguments, or result of call, or string

>From my understanding, `ast.arguments` and `inspect.Signature` are the
two builtin ways of defining the function name and arguments in a
structured way.

What I am creating is a way of configuring? well let's be specific to
my use-case. I am building a way to configure TensorFlow.
One which is very type-driven, and will error as early-as-possible
when incorrect types or lack of required parameters are provided.

I can dynamically infer things like so:
import inspect
import tensorflow as tf

sig = inspect.signature(tf.keras.optimizers.Adam)
    "default": sig.parameters[param].default,
    "name": param,
    "typ": type(sig.parameters[param].default).__name__
             if sig.parameters[param].default is not inspect._empty
and sig.parameters[param].annotation is inspect._empty
             else sig.parameters[param].annotation,
    for param in sig.parameters if param != 'name'

I can also parse the docstring, as I do in my doctrans library and tool.

Which will give me the options I can provide the class. So there's an
obvious solution, to generate these classes:
class TensorFlowConfig(object):
    optimizer: Optional[Optimizer] = None

# Or maybe a `Protocol`?
class Optimizer(object): pass

class AdamConfig(Optimizer):
    learning_rate: float = 0.001
    beta_1: float = 0.9
    beta_2: float = 0.999
    epsilon: float = 1e-07
    amsgrad: bool = False
    kwargs: dict = {}

TensorFlowConfig().optimizer = AdamConfig()

But, keeping in mind the goal to expose everything in all interfaces,
the question then becomes how to generate an argparse parser from
this. And how to generate a function from this. And how to ensure that
whatever code-completion interface one uses in Python, that it can
figure out what the correct parameters are.

So I should get an error about incorrect type when I:
--optimizer 'Adam' 'learning_rate = True'

# class*
TensorFlowConfig().optimizer = tf.keras.optimizers.Adam(learning_rate=True)
TensorFlowConfig().optimizer = AdamConfig(learning_rate=True)

# Function*

* both of these next two lines?after the heading?should be equivalent

Syntax like this should also be valid
TensorFlowConfig().optimizer = 'tf.keras.optimizers.Adam'
TensorFlowConfig().optimizer = 'Adam'


Do I have huge annotations like this*, with an `Any` lopped on for
further analysis down the line? - So it can easily generate into
`choices` for argparse? [which will need to be subclassed in order to
enable that "--optimizer 'Adam' 'learning_rate = True'" syntax]


What makes sense?

Thanks for your insights,

Samuel Marks
Charity <> | consultancy
<> | open-source <> |
LinkedIn <>

PS: This is the `doctrans` project I referenced