Skip to content

Conversation

@barikata1984
Copy link
Contributor

…ed to wandb.Table into a double-nested list

@barikata1984
Copy link
Contributor Author

confirmed

@barikata1984
Copy link
Contributor Author

barikata1984 commented Nov 7, 2023

Why:

  1. training curves, such as total_loss and rgb_loss, are not plotted properly on a synced wandb dashboard: only a single point was plotted on it
  2. log_table() for _WandB did not work with the following error:
  File "/home/atsushi/workspace/kaolin-wisp/app/nerf/main_nerf.py", line 128, in <module>
    trainer.train()  # Run in headless mode
  File "/home/atsushi/workspace/kaolin-wisp/wisp/trainers/base_trainer.py", line 371, in train
    self.iterate()
  File "/home/atsushi/workspace/kaolin-wisp/wisp/trainers/base_trainer.py", line 330, in iterate
    self.end_epoch()    # determines if optimization keeps running
  File "/home/atsushi/workspace/kaolin-wisp/wisp/trainers/base_trainer.py", line 308, in end_epoch
    self.validate()
  File "/home/atsushi/workspace/kaolin-wisp/wisp/trainers/multiview_trainer.py", line 294, in validate
    self.tracker.log_table("args", record_dict, self.epoch)
  File "/home/atsushi/workspace/kaolin-wisp/wisp/trainers/tracker/tracker.py", line 227, in log_table
    dashboard.log_table(caption, data, step)
  File "/home/atsushi/workspace/kaolin-wisp/wisp/trainers/tracker/tracker.py", line 453, in log_table
    table = wandb.Table(columns=data.keys(), data=data.values())
  File "/home/atsushi/miniconda3/envs/wisp/lib/python3.9/site-packages/wandb/data_types.py", line 290, in __init__
    self._init_from_list(data, columns, optional, dtype)
  File "/home/atsushi/miniconda3/envs/wisp/lib/python3.9/site-packages/wandb/data_types.py", line 309, in _init_from_list
    assert isinstance(data, list), "data argument expects a `list` object"
AssertionError: data argument expects a `list` object

What:

  1. Configure a wandb dashboard first if it is enabled because wandb.init() should be called earlier than instantiating SummaryWrite according to the official description of wandb (link)
  2. Cast columns argument into a list and data argument into a 2D list because the columns argument should be iterable and, both the data argument and its element should be iterable as well based on the source (link)

P.S. PR #183 was my fault. Sorry about that. Please ignore it

@barikata1984 barikata1984 reopened this Nov 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant