1. Mailing Lists
  2. Contributors
  3. Re: 14.0 branches

Contributors mailing list archives

contributors@odoo-community.org

Browse archives

Re: 14.0 branches

Re: 14.0 branches

Avatar

Re: 14.0 branches

by
Camptocamp France SAS, Alexandre Fayolle
- 12/10/2020 14:47:43
I'm happy with generated requirements.txt and oca_dependencies files. This will remove some errors caused by manual management of these files anyway.

Regarding oca_dependencies: I use them to easily find when an addon is defined, where to direct a contribution, etc. I also happened to "discover" some OCA repositories that way.

Regarding requirements.txt: the latest version of Odoo's runbot uses these to install the requirements. It will help using OCA repositories on Odoo.sh, and I think it is a valuable service we provide to our users.

Regarding pip installation of addons, I'm not sure this is available on odoo.sh, and I think this is a use case we should not underestimate.

Le lun. 12 oct. 2020 à 09:07, Mignon, Laurent <laurent.mignon@acsone.eu> a écrit :
HI Folks,

I think it's very important to keep the debate open and to present your arguments in a constructive way.I must admit that I am a little saddened to read accusations of half-truths when the process is meant to be open and constructive. It seems to me that the goal here is not to impose a new methodology but to confront our practices with those of the python community. The fact that the aim of this approach is to simplify the maintenance of our OCA repositories as well as to reduce the learning curve for new developers, I find this more than remarkable and valuable.

Since many integrators' tools/processes are now based on the oca_dependencies.txt and requirement.txt files, in view of the discussion it seems necessary that somehow these should be available again. It seems to me that automating the generation of these files would be more than beneficial because it would avoid inconsistencies in the future for all the processes based on them.

Over the years I have used different tools and methodologies to try to improve the management of our dev -> prod processes of our python projects. I personally contributed to buildout, imagined and wrote git-agggregator, .... The goal has always been to improve the traceability and reproducibility of our developments and deployments. Today, and thanks to the evolutions around pip, I'm happy to see that python tools allow me to improve and simplify these processes without having to use specific tools. (While bringing more freedom and features).

Is it possible to envisage an evolution of our integration mechanisms towards pip support while retaining the necessary elements for the other approaches developed by the various parties?

I find it motivating to confront this approach with all the use cases of each other in order to determine the limits of this approach. This will at least allow us to improve our knowledge and perhaps correct certain ideas.

Regards


On Mon, Oct 12, 2020 at 12:11 AM Nhomar Hernández <nhomar@vauxoo.com> wrote:


El dom., 11 de oct. de 2020 a la(s) 12:17, Stéphane Bidoul (stephane.bidoul@acsone.eu) escribió:
As said, my only goal is to improve the OCA contribution workflow.
Of course, I don't want to force any specific installation method onto integrators workflows.

Sorry my friend, but if you continue insists that the Integration, deployment and testing are separated and differently managed, then you are doing and proposing exactly the opposite here.

Everybody should deploy/test/use the software in the same unique way (the only difference should be the extra dependencies required for the testing environment itself that are not required in production, and even this can be used in production as well but some prefer not doing it.
 

So let's try to keep the discussion focused.

To be a little bit more specific, here are some aspects that the new method improves:

Less Redundancy

What is in requirements.txt must be added manually and is redundant to addons manifests. oca_dependencies.txt is also redundant and the information can be found elsewhere.

Why oca_dependencies is redundant? currently, Where else I can find that information?


Better testing of dependency declarations

We don't test external python dependencies declarations in manifests because we rely on a redundant, manually maintained requirements.txt

With PIP this will happen too! (that's the problem with the pip-hell that I linked my friend).
 
pylint-odoo helps but there are limits to what it can detect.

The same in the other way around, pylint-odoo was born because we need to test things that need odoo running and/or a very specific odoo environment.


Also, we publish wheels for OCA addons because it is by far the easiest for newcomers (and veterans alike I should say) to discover dependencies (addons and external python dependencies). Bad dependencies makes for a suboptimal user experience for newcomers installing the wheels.

I think the opposite here, (I am a veteran python consumir as you are, and honestly I prefer 100% read how to deploy odoo than deal with pip-hell).

But in this case you get a point which is the newcomers we should teach more and document better the current process, not remove features for them (it is dangerous in the long term).

 

Reduced blast radius when addons are introduced with specific test requirements, or "exotic" test requirements break

When an addon has specific test requirements it has side effects in all repos that transitively depend on its repo, whether or not the modules are actually needed. When installing dependencies at the addon level instead of the repo level, we drastically reduce the blast radius of such things. Examples: server_environment that required a mandatory option in odoo.cfg. Or an obscure dependency of some rarely used addon in server-tools that suddenly breaks on PyPI and impacts all repos that transitively depend on server-tools because they all end up installing all requirements.txt of server-tools. When that happens, the impact is usually so wide that everyone has to rush to find solutions to unlock the situation. If the impact is contained to the addon and those that depend on it, that buys us more time to find good solutions.

But this is only solved if we move the dependencies (the current one) to module level, not change all to PIP.

On the other hand, you get a point!, solution: Avoid tools that change the odoo.cfg modification as a good practice, but that's another story..

 

Finer grained approach to declaring unmerged dependencies

With the optional test-requirements.txt, you can declare override dependencies to target unmerged individual addons like this:

WAO:

How i that simplest that say:

folder git@url/repo branch  ?

 
This means you can reference several PRs of the same repos, or several addons of different PRs in another repo (I don't think one can do that with oca_dependencies.txt, where you reference whole repos).

Same as above.

[BTW, @Moises, to answer your question above you could use such references to your own repos. And it reduces the need to use git-aggregator too. - but I digress]

To close the list, I'd say dependency management is hard and relying on the broader python ecosystem to solve the problem can only help us in the long run (pip in particular is going to release a new sophisticated dependency resolver later this year).

I think we should wait for that, I have 5 years hearing this will happen from Python Foundation and it is more complicated than ever with the expansion of python. By that time we should wait for that PEP and understanding it before move all.

And Odoo is not particularly messy nor does it have anything very special in that matter.

So how can we progress?

For the reasons above, I'd like to move out of the status quo.
In the past the only arguments I had received were of the general class "I don't like it", but nothing really concrete.

From me you received the same arguments I am giving here... it is not that simple, I am fully worried, you are saying half/truths here, almost all the arguments are incomplete.

But that's another topic here.
 
So yeah, I guess I'm pushing a little bit in the hope of getting solid feedback.

Solid feedback:

Let's use pip BUT the old way too, until the PIP is stable and we can seriously have something solid.

I am fully worried, we end up with a Frank-PIP as well which will return us to the same point we are today but with PIP.

Your half/true statements to be fairly honest, shows that neither you or us will agree because neither we want to deal with PIP-hell until Odoo supports it, and you are not working with the current status Quo.

I think the blocking point is ther.

No harm is done (there are no cross-repo dependencies in v14 branches yet :) and as said, we can easily push an update to .travis.yml.

No man, there is.

We are deploying v14 since 4 month ago.
We have even saas-XX repos preparing thing to be migrated.

Tis is not something that must be done in September pre-experience.

Let' s work for version 15 from now on (or even 16) to prepare the future.

With the new fast migration strategy in Odoo, we can not make this change now, for december we will have a ton of migrated databases and left OCA behind too much.

And the good thing is that the feedback is coming:
- At this point I see Moises who says it relies on oca_dependencies.txt in its workflow.
- Daniel mentioned a documentation value of oca_dependencies.txt.

A few questions:
- @Moises, do you rely on requirement.txt too in your own workflow ?

Yes, we do.
- Do others rely on it ? (for instance, does Dooba need oca_dependencies.txt, requirements.txt in OCA repos ?)

Yes, we do!

If we conclude that oca_dependencies.txt and/or requirements.txt must be preserved in all repos in v14 (and I'm totally fine with that), we can then discuss how best to do it.

Not just preserve it (it is not a matter of have a file man, please I ask you to not oversimplify this) All the CI is and will rely on such files too!, if not you are forcing people to re-write such information in a dummy file (PIP-s way) please..

 
Either manual with e.g. PIP mode on Odoo and OCA mode on OCB as Moises suggested - but that would be a burden for contributors to understand and maintain both approaches. Or try to generate oca_dependencies.txt and requirements.txt, which should be feasible at first sight, but that's more work. Or keep the status quo :/
The easiest way is if you update your workflow instead try to change the current one to something that will left all incomplete....

But again on my side I think or both or the current one.
 

Cheers,

-sbi

regards!

 


On Sun, Oct 11, 2020 at 7:06 PM Stéphane Bidoul <stephane.bidoul@acsone.eu> wrote:
On Sun, Oct 11, 2020 at 5:41 PM Ivan Todorovich <ivan.todorovich@druidoo.io> wrote:
Would testing with PIP also catch compatibility errors between modules that don’t depend on each other? Or errors with modules that are in the upstream chain of dependencies?

@Ivan the new method does not change what is installed on the test database. What changes is only the list of addons present in the addons path. So the scenarios you mention should work just like before.

-sbi

_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe



--

Nhomar G Hernández

Vauxoo | CEO

¡Construyamos algo genial!
Cel: +52 (477) 393.3942 | Telegram: nhomar | Twitter: @nhomar

México · Venezuela · Costa Rica · Perú

phone nhomar@vauxoo.com phone vauxoo.com/contactus  

_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe

_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe



--
Alexandre Fayolle
Chef de Projet
Tel : + 33 (0)4 58 48 20 30

Camptocamp France SAS
http://www.camptocamp.com

Reference