What to do when sample size is too small during usability study?

If the sample was just a bit too small to achieve statistical significance in a remote usability study(confidence intervals too wide due to more variability than expected)

Is it acceptable to just run more users, adding a second "leg" to the study and treating them as one data set, to try and narrow the interval? (Presuming no changes to the recruiting method or the website)

Note: data collected is both binomial (task pass/fail) and continuous (task times)