1. Mailing Lists
  2. Contributors
  3. Re: 14.0 branches

Contributors mailing list archives

contributors@odoo-community.org

Browse archives

Re: 14.0 branches

Re: 14.0 branches

Avatar

Re: 14.0 branches

by
Acsone SA/NV, Denis Roussel
- 12/10/2020 12:17:49
Hi Nhomar,

  • SHA management with git is fully supported: pip install <addon-name> <git-url>@<sha>#subdirectory=<setup dir>
  • didn't catch second point.




On Mon, Oct 12, 2020 at 12:07 PM Nhomar Hernández <nhomar@vauxoo.com> wrote:
Hello Laurent and all.


El lun., 12 de oct. de 2020 a la(s) 02:07, Mignon, Laurent (laurent.mignon@acsone.eu) escribió:
HI Folks,

I think it's very important to keep the debate open and to present your arguments in a constructive way.I must admit that I am a little saddened to read accusations of half-truths when the process is meant to be open and constructive. It seems to me that the goal here is not to impose a new methodology but to confront our practices with those of the python community.
Me too!

But I think what we are having now is a "coup d'etat" and nobody in charge is answering (at least what I see) the questions.

I want and will be Open and argumenting constructively, but to be fairly honest, 3 things are clear for me we can not do with pip (and I am not moving forward until those are not solved and the reasons all was born, I can not be constructive when somebody come to brake everything is built even with the best of the intentions).

1.- No SHA management present in PIP linked with git (or if it is an extra layer BTW).
2.- No Oca dependencies (remember it is not optional nor auto done, We need to have one and only one place where to say technically where is my dependency code)  , which means if PIP consume oca_dependencies then nice, but not make us to re-create all into PIP and then auto create a oca_dependency.txt


I have an honest question:

Do you think Odoo will be pip-installable in some moment?

I see communities like tryton and plone that claim be so so so pip lovers and python perfectionists, and I personally have FAST way to consume (with PIP) but **really complex way** to contribute, because PIP is a tool to improve the **pull** process not the **push** process.

Why do we insist on changing all to PIP when we have a flow that works!? and in fact, each time I take a customer that deploys with PIP they have a perfect mess, nobody knows anything about nothing. 



 
The fact that the aim of this approach is to simplify the maintenance of our OCA repositories as well as to reduce the learning curve for new developers, I find this more than remarkable and valuable.

Since many integrators' tools/processes are now based on the oca_dependencies.txt and requirement.txt files, in view of the discussion it seems necessary that somehow these should be available again. It seems to me that automating the generation of these files would be more than beneficial because it would avoid inconsistencies in the future for all the processes based on them.

I need to say man to be honest, there is no proof that if we change to PIP more people will contribute (there is non a single proof this will happen), In fact I think it will increase the consumption of modules but it will decrease the actual contribution.

Why?

Because you are separating process from develop-deploy-consume on different tools then this will be impossible to use the same effort to contribute and people with come down with the contribution.

 

Over the years I have used different tools and methodologies to try to improve the management of our dev -> prod processes of our python projects. I personally contributed to buildout, imagined and wrote git-agggregator, .... The goal has always been to improve the traceability and reproducibility of our developments and deployments. Today, and thanks to the evolutions around pip, I'm happy to see that python tools allow me to improve and simplify these processes without having to use specific tools. (While bringing more freedom and features).

And me too!

But what I do not know how to explain myself and it looks like I am alone on this battle is:
image.png

PIP is not a tool to **contribute** with the code but to **distribute** the code.

Then, it will not improve the collaboration, it will increase (and it is nice) the consumption of the software, but say that other pythonists will come easier because PIP is not true.

IMHO this should be the approach (as any other package).

code -> oca_dependencies > download dependencies -> test packages IF Ok then generate the new version for a pIP package test the PIP package and END.

What I understand (and please correct me if I am wrong) is that you want to start to maintain PIP packages and ignore completely the current approach, and I can't find a way or a single reason why loose the flexibility that a single file gives us (the opca_dependencies) as centralizer.

Note: I am pretty sure the current mechanism has other issues but that in particular for me is the no-go topic.

 

Is it possible to envisage an evolution of our integration mechanisms towards pip support while retaining the necessary elements for the other approaches developed by the various parties?

I am IN but not on this way, let's at least make a plan together and inform a proper time (not overnight when a version which almost all modules will work from the old one was just released).

I find it motivating to confront this approach with all the use cases of each other in order to determine the limits of this approach. This will at least allow us to improve our knowledge and perhaps correct certain ideas.

Me too!

 

Regards

I ask for apologize If I am being rude, but I can't find other ways to be clear/plain and simple about my point.

Regards.


 


On Mon, Oct 12, 2020 at 12:11 AM Nhomar Hernández <nhomar@vauxoo.com> wrote:


El dom., 11 de oct. de 2020 a la(s) 12:17, Stéphane Bidoul (stephane.bidoul@acsone.eu) escribió:
As said, my only goal is to improve the OCA contribution workflow.
Of course, I don't want to force any specific installation method onto integrators workflows.

Sorry my friend, but if you continue insists that the Integration, deployment and testing are separated and differently managed, then you are doing and proposing exactly the opposite here.

Everybody should deploy/test/use the software in the same unique way (the only difference should be the extra dependencies required for the testing environment itself that are not required in production, and even this can be used in production as well but some prefer not doing it.
 

So let's try to keep the discussion focused.

To be a little bit more specific, here are some aspects that the new method improves:

Less Redundancy

What is in requirements.txt must be added manually and is redundant to addons manifests. oca_dependencies.txt is also redundant and the information can be found elsewhere.

Why oca_dependencies is redundant? currently, Where else I can find that information?


Better testing of dependency declarations

We don't test external python dependencies declarations in manifests because we rely on a redundant, manually maintained requirements.txt

With PIP this will happen too! (that's the problem with the pip-hell that I linked my friend).
 
pylint-odoo helps but there are limits to what it can detect.

The same in the other way around, pylint-odoo was born because we need to test things that need odoo running and/or a very specific odoo environment.


Also, we publish wheels for OCA addons because it is by far the easiest for newcomers (and veterans alike I should say) to discover dependencies (addons and external python dependencies). Bad dependencies makes for a suboptimal user experience for newcomers installing the wheels.

I think the opposite here, (I am a veteran python consumir as you are, and honestly I prefer 100% read how to deploy odoo than deal with pip-hell).

But in this case you get a point which is the newcomers we should teach more and document better the current process, not remove features for them (it is dangerous in the long term).

 

Reduced blast radius when addons are introduced with specific test requirements, or "exotic" test requirements break

When an addon has specific test requirements it has side effects in all repos that transitively depend on its repo, whether or not the modules are actually needed. When installing dependencies at the addon level instead of the repo level, we drastically reduce the blast radius of such things. Examples: server_environment that required a mandatory option in odoo.cfg. Or an obscure dependency of some rarely used addon in server-tools that suddenly breaks on PyPI and impacts all repos that transitively depend on server-tools because they all end up installing all requirements.txt of server-tools. When that happens, the impact is usually so wide that everyone has to rush to find solutions to unlock the situation. If the impact is contained to the addon and those that depend on it, that buys us more time to find good solutions.

But this is only solved if we move the dependencies (the current one) to module level, not change all to PIP.

On the other hand, you get a point!, solution: Avoid tools that change the odoo.cfg modification as a good practice, but that's another story..

 

Finer grained approach to declaring unmerged dependencies

With the optional test-requirements.txt, you can declare override dependencies to target unmerged individual addons like this:

WAO:

How i that simplest that say:

folder git@url/repo branch  ?

 
This means you can reference several PRs of the same repos, or several addons of different PRs in another repo (I don't think one can do that with oca_dependencies.txt, where you reference whole repos).

Same as above.

[BTW, @Moises, to answer your question above you could use such references to your own repos. And it reduces the need to use git-aggregator too. - but I digress]

To close the list, I'd say dependency management is hard and relying on the broader python ecosystem to solve the problem can only help us in the long run (pip in particular is going to release a new sophisticated dependency resolver later this year).

I think we should wait for that, I have 5 years hearing this will happen from Python Foundation and it is more complicated than ever with the expansion of python. By that time we should wait for that PEP and understanding it before move all.

And Odoo is not particularly messy nor does it have anything very special in that matter.

So how can we progress?

For the reasons above, I'd like to move out of the status quo.
In the past the only arguments I had received were of the general class "I don't like it", but nothing really concrete.

From me you received the same arguments I am giving here... it is not that simple, I am fully worried, you are saying half/truths here, almost all the arguments are incomplete.

But that's another topic here.
 
So yeah, I guess I'm pushing a little bit in the hope of getting solid feedback.

Solid feedback:

Let's use pip BUT the old way too, until the PIP is stable and we can seriously have something solid.

I am fully worried, we end up with a Frank-PIP as well which will return us to the same point we are today but with PIP.

Your half/true statements to be fairly honest, shows that neither you or us will agree because neither we want to deal with PIP-hell until Odoo supports it, and you are not working with the current status Quo.

I think the blocking point is ther.

No harm is done (there are no cross-repo dependencies in v14 branches yet :) and as said, we can easily push an update to .travis.yml.

No man, there is.

We are deploying v14 since 4 month ago.
We have even saas-XX repos preparing thing to be migrated.

Tis is not something that must be done in September pre-experience.

Let' s work for version 15 from now on (or even 16) to prepare the future.

With the new fast migration strategy in Odoo, we can not make this change now, for december we will have a ton of migrated databases and left OCA behind too much.

And the good thing is that the feedback is coming:
- At this point I see Moises who says it relies on oca_dependencies.txt in its workflow.
- Daniel mentioned a documentation value of oca_dependencies.txt.

A few questions:
- @Moises, do you rely on requirement.txt too in your own workflow ?

Yes, we do.
- Do others rely on it ? (for instance, does Dooba need oca_dependencies.txt, requirements.txt in OCA repos ?)

Yes, we do!

If we conclude that oca_dependencies.txt and/or requirements.txt must be preserved in all repos in v14 (and I'm totally fine with that), we can then discuss how best to do it.

Not just preserve it (it is not a matter of have a file man, please I ask you to not oversimplify this) All the CI is and will rely on such files too!, if not you are forcing people to re-write such information in a dummy file (PIP-s way) please..

 
Either manual with e.g. PIP mode on Odoo and OCA mode on OCB as Moises suggested - but that would be a burden for contributors to understand and maintain both approaches. Or try to generate oca_dependencies.txt and requirements.txt, which should be feasible at first sight, but that's more work. Or keep the status quo :/
The easiest way is if you update your workflow instead try to change the current one to something that will left all incomplete....

But again on my side I think or both or the current one.
 

Cheers,

-sbi

regards!

 


On Sun, Oct 11, 2020 at 7:06 PM Stéphane Bidoul <stephane.bidoul@acsone.eu> wrote:
On Sun, Oct 11, 2020 at 5:41 PM Ivan Todorovich <ivan.todorovich@druidoo.io> wrote:
Would testing with PIP also catch compatibility errors between modules that don’t depend on each other? Or errors with modules that are in the upstream chain of dependencies?

@Ivan the new method does not change what is installed on the test database. What changes is only the list of addons present in the addons path. So the scenarios you mention should work just like before.

-sbi

_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe



--

Nhomar G Hernández

Vauxoo | CEO

¡Construyamos algo genial!
Cel: +52 (477) 393.3942 | Telegram: nhomar | Twitter: @nhomar

México · Venezuela · Costa Rica · Perú

phone nhomar@vauxoo.com phone vauxoo.com/contactus  

_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe

_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe



--

Nhomar G Hernández

Vauxoo | CEO

¡Construyamos algo genial!
Cel: +52 (477) 393.3942 | Telegram: nhomar | Twitter: @nhomar

México · Venezuela · Costa Rica · Perú

phone nhomar@vauxoo.com phone vauxoo.com/contactus  

_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe



--
__________________________________________
Denis Roussel
Software Engineer
Acsone SA, Succursale de Liège (Val Benoît)
Tel    : +32 2 888 31 49
Fax   : +32 2 888 31 59
Gsm : +32 472 22 00 57

Acsone sa/nv
Boulevard de la Woluwe 56 Woluwedal | B-1200 Brussels  | Belgium
Quai Banning, 6 (Val Benoît) | B-4000 Liège | Belgium
Zone Industrielle 22 | L-8287 Kehlen | Luxembourg

Reference