Automated testing

[mgalante]

Hi folks!

Are we considering any automated test of the source code?
As the source code starts to grow up it would be difficult to be sure that everything is working as expected...

What do you think?

[getify]

Fair question. I have mixed feelings about that, given some of the unique characteristics of this project. In particular, there's a bit of a complicated interchange between the JS of the page, the JS in a web worker, and a WASM module (doing the Argon2 password hashing). Throw in that we're storing data in sessionStorage and IndexDB, and we've got a situation that will be difficult to do much in the way of useful automated unit tests on.

It would require, most likely, an automated integration test in some scriptable browser environment (not like a virtualized DOM in Node), perhaps Puppeteer or PhantomJS or something like that.

I think eventually that might in fact be useful, but so much about about the app's interface and interaction is going to change, if we set out a bunch of specific integration tests right now, those tests are going to be constantly either broken or needing update, something that may slow down dev.

So it's a bit of a difficult tradeoff. But I'm certainly open to feedback/discussion here.

[adrichey]

I feel like it could be a hybrid approach. I think keeping the barrier to entry low is good in theory, but it may back us into a corner security-wise in the long run if we can't trust that the individual pieces are working as intended. My two cents would be that if people want to contribute without tests, that is fine and we can hope that integration tests will catch things. But maybe those who want to include tests, they could do so over the smaller pieces they are building in Node JS tests (requires v18.5.0). This of course excluding things that need a DOM to verify until the leadership on the project decides on what they want to do there (mocking DOM through jsdom or something, how they want to integrate, etc.).

That being said, I would love to see us get integration tests in over at least the authentication at some point soon, personally. I would also be more than happy to get something put together on that front if we want. I have a ton of experience building out testing tools. If we could get on node v18.5.0 I could build the integration suite tools using minimal dependencies as well (mainly just Puppeteer and Node). Though I can work with the v16 confines if need be. It might just require more dependencies, which I know we are trying to avoid.

In any case, if there is ever a call or something around this to talk solutions I would be glad to help. :) Thanks for listening!

[getify]

excluding things that need a DOM

It's not just DOM as it relates to UI presentational elements. As I described above, most of the important functionality right now relies on the asynchronous interchange of data between the page, the Worker, an inline WASM module, and IndexedDB. These are all things that have very particular behaviors in a browser that are, to say the least, not the same in Node (even with stuff like JSDom).

I strongly doubt we'll be able to test most of this existing code, in any meaningful way, in Node. That's why I think it's a premature optimization to start implementing required Node-focused unit tests.

I am quite open to the importance of using tests for development. I do myself, for sure. I am just not quite convinced (yet) that these tests need to become persistent artifacts of the project (again, yet) that all developers share, and thus share the burden of setting up, maintaining, adding to, etc.

So what I'm getting at is, if you have code you're writing that you want to also write tests for, please do so. But let's hold off on assuming that we're creating a centralized unit-test suite that spans the whole project. Those tests could just remain local to your forked branch and not necessarily be merged into the main core.

I'd rather we eventually put the majority of our testing effort into a full-fidelity scripted browser integration testing system, once we have a bit better sense of how the UI/UX of the app is starting to shape up.

[getify]

If we could get on node v18.5.0

I have not upgraded my server, which is currently hosting the live app demo, to node 18. I tried an upgrade a month ago, and something weird broke in one of my other apps, so I rolled it back until I had time to investigate.

Obviously I plan to upgrade that server to latest node eventually, once I get the time to do so. But if we start building anything server-wise that assumes Node 18, deployment to the live server could fail in some surprising way. So that's why I said "assume Node 16" for now.

That said, that doesn't mean at all that your local Node has to be 16. If you have 18 and want to use it for your stuff, that's perfectly fine and great. Just be careful if any of that starts to create code that needs to run out on my server, as the upgrade from 16 to 18 is blocked for the time being.

[adrichey]

Makes sense. Glad it is something on your radar. I am starting to get a feel for how these pieces connect and I am in agreement. I think an integration approach will be a good way to go with a staging environment once things get a little more functional. Good call on that front, for sure. Thanks for explaining your logic further. I also wrote my reply before really diving into the codebase and thinking through how all this will work together conceptually.

[getify]

What do you all think?

https://dev.to/polterguy/unit-testing-frontend-code-is-probably-useless-27n9