Contributors mailing list archives
contributors@odoo-community.org
Browse archives
Re: demo data and tests
by
Acsone SA/NV, Laurent Mignon
Hi
On my projects tests are run at 2 levels:
- At development level: The developer tests the code he has written to ensure that it works as expected. For this, the developper writes unit tests that are run on an empty database with only this addon installed. This ensures that the code is correct and works properly in the context defined by the dependencies of the addon.
- At integration level: The CI runs all the tests on a database with all the addons installed. This ensures that the code is correct and works properly in the context of the whole system.
To achieve this, we apply the following guidelines:
- Tests don't rely on demo data; This eases the maintenance of the tests and ensures that the tests are not impacted by changes in the demo data. Moreover, it helps the developer to understand the context in which the tests are run.
- Code that modifies / constrains an existing behavior must be disabled by default and must be enabled by a configuration parameter. This ensures that if you run the tests from other addons on a database where this addon is installed, the tests will not be impacted by this addon. It also ensures that if the tests are run at insall and another addon modifies the same behavior, the tests will not be impacted by this addon if it has been installed previously in the installation process. (The disabled behaviour is activated at test setup). All the disabled features are enabled in a top level addon which is excluded from the installation by the CI and installed only into the client database.
- Some specific test addons are developed for specific integration tests. These addons are useful to ensure that addons installed side by side are compatible even if they don't depend on each other. At test setup, we first activate the features of the addons that are activable by configuration parameter.
- To avoid the need to use the "post_install' annotation, the CI runs the tests with pytest-odoo. The main difference compared to the way odoo runs the tests is that the registry is populated with all the addons installed into the database. pytest-odoo is a test runner only while when you run the tests with odoo, the tests are run along the installation/upgrade of the addons at the same time the registry is populated. To run tests with pytest-odoo, the first requirement is to have the addons to test installed in the database. One advantage of this approach is that even if your addon creates a new record for which another addon has created a new required field with a default value the tests will run correctly even if you don't depend on this addon.
- ...
We are still not perfect and since all the code modifying an existing behaviour is activated in production, some tests will fail if they are run on the production database since they are not designed to run with the modified behaviour.. But we solved a lot of issues we encountered in the past when tests were run by odoo at install/upgrade time.
Regards,
lmi
Use post install test tags and remove at install. And refuse PRs with required fields that don't set defaults. This is the normal practice anyway as you would get errors with or without using demo data.
Also, one other benefit, particularly in number based tests is we use a library called hypothesis to determine values and try to find failing combos.Le ven. 13 déc. 2024, 04:52, Raphaël Valyi <notifications@odoo-community.org> a écrit :Hello Odoo community,More and more the OCA contributors avoid writing tests which depend on demo data and so does Odoo SA. This makes it easier to ensure tests are simple and will run no matter how demo data might be touched by other modules.But eventually this could bring other benefits such as the ability to run these standard tests in the CI with production database dumps (that is where the demo data cannot be loaded in general).In this case, I have a first question: probably some customization's might add required fields. What would be the best practice to be able to run these "native" or OCA tests that might fail to create records with such new constraints? We could imagine some monkey patches and standard hooks to help... We kind of face this already today by running tests with all repo modules installed, but with production dumps, much more modules would be installed together along with some non OCA customization which also need to be developed more cheaply...But more important is my second question: if we set it as a good practice that OCA tests shouldn't depend on demo data, why are we still running the OCA CI with demo data enabled (still the case with Odoo 18 CI) ? Or is it planned we stop doing this in Odoo 19? I mean we could also save a good deal of server resources/time by not loading the gazillions of catchy Odoo CE demo data in each test run...Thank you very much for your answers.
--Raphaël ValyiFounder and consultant_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe
Laurent Mignon
Technical lead / Management Team
T: +32 2 8883148
Atrium Building, Drève Richelle 167 | B-1410 Waterloo | Belgium
Val Benoit, Quai Banning 6 | B-4000 Liège | Belgium
Zone industrielle 22 | L-8287 Kehlen | Luxembourg
