You're not doing anything wrong. Its sort of a caching issue.
When you change a template, by adding or changing a widget the change just doesn't immediately gets reflected. If you republish a page and visit it, you're essentially forcing a refresh which picks up the changes from the db.
It's annoying yes, but you're not doing anything wrong...