What is the standard/best way to call javascript (for Colorpickers, Media, …) within custom widgets, that work within the customizer as well?

I have created a custom widget that has two color settings. The color settings are based on this code https://gist.github.com/rodica-andronache/54f3ea95bcaf76435e55

The color pickers work fine, however, only after the customizer is saved and the page is reloaded. If you click on the color field straight after adding it to the panel, nothing happens.

My suspicion is that the javascript files are not firing at the right moment. I have tried enqueueing the files using admin_enqueue_scripts and customize_controls_enqueue_scripts and it doesn’t make a difference, the color pickers only work after reloading the page. What would be the standard way of enquing to make sure the javascript is fired at the right time?

Any help would be much appreciated. Thanks.