How to use argparse and yaml module on jupyter at the same time

In [1]:

import sys, os, argparse, yaml

How to use argparse and yaml

Basic usage of argparse in jupyter notebook

When using argparse module in jupyter notebook, all required flag should be False.
Before calling parser.parse_args(), we should declare as follows.

sys.argv = ['-f']

In [2]:

parser = argparse.ArgumentParser()
parser.add_argument('-config', help="configuration file *.yml", type=str, required=False, default='config.yml')
parser.add_argument('-args', help="learning rate", type=bool, required=False, default=False)
# training parameters
parser.add_argument('-epochs', help="num of epochs for train", type=int, required=False, default=100)
parser.add_argument('-lr', help="learning rate", type=float, required=False, default=0.00005)
parser.add_argument('-batch_size', help="batch size", type=int, required=False, default=64)
sys.argv = ['-f']
args = parser.parse_args()

Use yaml and argparse at the same time

If you want to know about Loader option in pyyaml, see this document in StackOverflow

In [3]:

%%writefile config.yml
epochs: 100
lr: 0.001
batch_size: 50

Overwriting config.yml

In [4]:

parser = argparse.ArgumentParser()
parser.add_argument('-config', help="configuration file *.yml", type=str, required=False, default='config.yml')
parser.add_argument('-args', help="learning rate", type=bool, required=False, default=False)
# training parameters
parser.add_argument('-epochs', help="num of epochs for train", type=int, required=False, default=100)
parser.add_argument('-lr', help="learning rate", type=float, required=False, default=0.00005)
parser.add_argument('-batch_size', help="batch size", type=int, required=False, default=64)
sys.argv = ['-f']
args = parser.parse_args()

if not args.args:  # args priority is higher than yaml
    opt = vars(args)
    args = yaml.load(open(args.config), Loader=yaml.FullLoader)
    opt.update(args)
    args = opt
else:  # yaml priority is higher than args
    opt = yaml.load(open(args.config), Loader=yaml.FullLoader)
    opt.update(vars(args))
    args = opt
print("arguments: {}".format(str(args)))

arguments: {'config': 'config.yml', 'args': False, 'epochs': 100, 'lr': 0.001, 'batch_size': 50}