Contributors mailing list archives

contributors@odoo-community.org

Browse archives

Avatar

Re: anybox buildout recipe and OCA

by
dar@devco.co
- 21/10/2015 17:08:48
The ideal focus is run a command similar to:
MAGIC_COMMAND git_url_project git_branch_or_pull_request
For all environments.

+10000 I'm so much committed to that, it hurts.. :)

@Raphael and Eric and other into docker: I would really have this discussion on a PR, because it is way easier to follow and track and organize knowledge. It's way easier to communicate this way. Please...

Saludos Cordiales
David Arnold

David Arnold BA HSG / Analista
315 304 13 68/ dar@devco.co

devCO - empresa de consultoría de sistemas (en fundación)
http://www.devco.co

This e-mail message may contain confidential or legally privileged information and is intended only for the use of the intended recipient(s). Any unauthorized disclosure, dissemination, distribution, copying or the taking of any action in reliance on the information herein is prohibited. E-mails are not secure and cannot be guaranteed to be error free as they can be intercepted, amended, or contain viruses. Anyone who communicates with us by e-mail is deemed to have accepted these risks. devCO is not responsible for errors or omissions in this message and denies any responsibility for any damage arising from the use of e-mail. Any opinion and other statement contained in this message and any attachment are solely those of the author and do not necessarily represent those of the company.


2015-10-21 9:38 GMT-05:00 Raphaël Valyi <rvalyi@akretion.com>:
Eric,

OK so suppose we want to collaborate for a base "production" oriented stack (because it s simpler). There are still issues:
  1. as you see on my prod Dockerfile, it's really trivial and simple to maintain. Is there any benefit in collaborating at this Dockerfile level? (vs the need of custonization, like I need a BR locale, you don't etc...)
  2. my Dockerfile is really short because I start from Heroku Cedarsih. But you don't. One can start directly from Ubuntu Trusty or even Debian. That'll give you a smaller image while mine will offer you opiniated but killer deploy tooling. All choice are defendable. So it't likely not everybody will agree on a base image and I don't think the OCA is entitled to impose such a choice. That means at best we agree on what list of packages should be applied atop of some custom base image and that list would have to be different or say modularized according to what your base image is... Again, does it leave enough room for collaboartion vs the weight of having OCA level consensus. I DOUBT.
  3. will that stack be made fro the Buildout recipe? My preference would be yes as this is now the tool blessed by the OCA. This is what we use in our Akretion Docker images, but not what you or Vauxoo are using... Will us have an agreement for using it?
  4. Suppose we agree on having a Docker oriented recipe. Will us have a simple image where applying a recipe will take like 5 minutes or a Dockerfile like ours with pre-fetched eggs (for versions 7,8,9?) makes it a 30 seconds run? Will us agree on our prefetch scripts despite they are totally hacky?
  5. As I explained at Akretion we opted for leaving the Odoo git repo OUTSIDE from the image. Because either you would put the full git repo + the OCB branches and that would be around 3Go or 4Go if I remember well. Either we would embeded a shallow repo, but if we want to support 3 versions of Odoo + OCB, that's still 6X the size of the code tree and may be around 1.5 Go... As a running instance will have to have the code tree checkouted at version X, at Akretion we opted for letting that only tree on the host (possibly shared accross projects such as in Voodoo but not in prod). Will us agree on that? That probably doesn't fix your China firewall issue... And if you need 6 different images for 7,8,9 and their OCB branches, then you didn't saved any space on the host either right...

So you see, even for a "basic" production image, that's a lot of very opiniated choice to be made. I personnaly don't want people that barely know Docker or Devops lobby to impose their view so that the OCA would prmote a dubious choice as the blessed solution and kill the real innovation happening outside of the OCA. Spcially many people of the audience are ERP or Python specialist but have no devops knowledge at all and easily believe people who have just a bit more knowledge as them no matter crappy the solutions they propose are. That's why I would prefer the OCA focus on other topics. That doesn't mean some people will not collaborate on some devops tools as it always happen with open source anyway.

Regards.


On Wed, Oct 21, 2015 at 7:23 AM, Eric Caudal <eric.caudal@elico-corp.com> wrote:

Hi Raphael,
Thanks for your feedback!
I think having simple/ready to use docker for production like you suggest is the way to go (with/without stack is not so important as long as this is solid and maintainable).
I agree with you about the different levels with the distinction (voodoo for dev, Docker for prod)
I personally would go:
- docker-compose/voodoo with postgres for dev
- Simple docker (no pg) that could be used as based for deployment (as is or not)
- Complex scripted docker to allow dependencies on the fly and/or (separately) complex pg setup.
But honestly I would let the community organize itself and if there is a maintainer for solution XYZ, it will flourish naturally.
Besides there is actually an ongoing work on anybox which is complementary to the odoo/docker

--
Eric Caudal [Founder and CEO]
Skype: elico.corp. Phone: + 86 186 2136 1670 (Cell), + 86 21 6211 8017/27/37 (Office)
Elico Shanghai (Shenzhen/Singapore) Odoo Gold Partner, best Odoo Partner 2014 for APAC
On 10/21/2015 12:38 PM, Raphaël Valyi wrote:
<blockquote cite="mid:CAByrsx3Z9Qxp+aaL1gt8aCqw5=b+CZHDnspZ6D0V4G00-63Teg@mail.gmail.com" type="cite">
@Eric,

also just for information, for production we use this simple stack based on Heroku Cedarish:
forget the 2 last lines and locale-gen pt_BR.UTF-8 that could go in a sub-image.


So it means at Akretion, we have:

                               Devstep - Voodoo    (for dev)
                             /
Heroku Cedarish
                             \
                               Herokuish - Odoocker  (for prod, no Postgres inside)


These two Dockerfile are opiniated toward pre-fetching the recipe eggs (using the real setup.py of Odoo 7, 8 and 9) so that to spin Odoo on it you just need to run the bin/buildout of the Anybox Recipe (which is fast provided you set a few symlinks to the cached eggs. But that's a choice that is quite compatible with the OCA choice of using the Anybox recipe...

What is added atop of Heroku Cedarich (here behind herokuish) could be shared with others and OCA or not but that would assume people are ok to build atop of Cedarish (not what you do for instance).
But then that would be just 30 lines of shared bash including the eggs pre-fetching scripts for Odoo 7, 8 and 9. IMHO not worth making noise at the OCA for 30 lines of bash...

That would define a simple base image able to run Odoo (like you do without pre-fetching the recipe eggs and without using Cedarish).

In my opinion, at this stage any tool, aiming at doing more than that will assume very strong opinions such as;
  • docker compose at leats for dev?
  • or embedded posgres for dev?
  • production image instead?
  • way to deal with the postgres and other config params: dinosaur like Odoo or 12 factors instead?
  • Docker launcher technology and it portability (Bash, Python, Golang...) ?
and I think it's way to early to say some of these approaches are standard enough at the OCA level before things are battle tested by a community before. And also as I said different tools will serve different needs and different integrator businesses. That would be very pretentious to claim for the silver bullet container when this field see so much innovation today.
In fact this field evolve at a speed I don't believe the OCA would ever follow. That's why I say the OCA should focus on Odoo modules and now the recipe, nut anything past this is really a lot of trial and error by nature...


Regards.

On Tue, Oct 20, 2015 at 10:38 PM, Eric Caudal <eric.caudal@elico-corp.com> wrote:

Hi Rafael,
I dont intend to store any docker in OCA: just docker files and scripts as you suggest.
There might be several docker files level with different complexity or customization or platforms. We might want to regulate it a bit but we can see that later.
I like the docker with bare odoo but having a docker+postgres (like voodoo) is a nice alternative (all-in-one windows like).
We will work with XCGD (and any contributor) on proposing solution and directions for contribution.
Best regards

--
Eric Caudal [Founder and CEO]
Skype: elico.corp. Phone: + 86 186 2136 1670 (Cell), + 86 21 6211 8017/27/37 (Office)
Elico Shanghai (Shenzhen/Singapore) Odoo Gold Partner, best Odoo Partner 2014 for APAC
On 10/20/2015 11:37 PM, Raphaël Valyi wrote:
<blockquote cite="mid:CAByrsx2aBZiuvoidTo1xjkgpswP_jqur-q4cyEHKi3V9=O7BSw@mail.gmail.com" type="cite">
Hello Eric,

I think we wanted the same simplicity goal as you with Voodoo. Still we made it complete enough so it really supports our own complex projects at Akretion too... (like we really like pinning revision using the Anybox recipe, and it's our deliverable lingua franca as I explained etc...).

You have this firewalll issue in China. Here in Brazil our issue is expensive servers. In France servers are so cheap that their cost don't matter at all and the economy is around customized projects. So you see, every use case will call different solutions. At the end of the day, our Dockerfiles will be different and reflect our very specific needs...

Now what could be maintained at the OCA IMHO are small scripts that could be used inside these Dockerfiles, But I'm not even sure...
So for instance here are the small scripts we use to pre-fetch the recipe eggs in the image to make the build a breeze at the developer environment: https://github.com/akretion/voodoo-image/tree/master/stack/build
We reuse that script both and the dev and production Docker images for instance.
But may be even that is already too specific; I won't even submit that to the OCA (and it's hacky too).

You may suggest maintaining the list of Debian dependencies packages. But as we start our stacks for Heroku Cedarish we would use only a fraction of it, so...

May be our dev Docker images will make use of OCA scripts like to lint repos or launch OCA tests (or test the project in a containairized Runbot)...

But overall, I see it more like we will share the libs used by our Docker tools inside the OCA, not the final Docker tools themselves, at least until there is some consolidation outside of the OCA (if that happens with Voodoo, that wouldn't be a problem on our side moving it to the OCA, but the plebiscite should happen first, just like it happened with Anybox Recipe).

Regards.



On Tue, Oct 20, 2015 at 5:08 AM, Eric Caudal <eric.caudal@elico-corp.com> wrote:

Hi Rafael,
I will have to have a look at Voodoo ;).
My idea of docker is actually to keep it simple to allow users to have a solid docker they can rely on without needing a high level of install/setup. Only few commands and go.
The second point is to allow the build of OCA docker repository, always up to date and always available.
We have built a docker file that allows oca_requirements.txt usage so that users can easily test repos.
This kind of features will probably satisfy 95% of the public and allow easy discovery of Odoo/OCA without polluting environment.
NB: In our specific case (China), we have to struggle as well with mirroring and docker mirroring works fine (from > 12h to download a docker from abroad to 3 min when using a mirror).
Anyway I agree with your approach and several tools might meet several requirements.

--
Eric Caudal [Founder and CEO]
Skype: elico.corp. Phone: + 86 186 2136 1670 (Cell), + 86 21 6211 8017/27/37 (Office)
Elico Shanghai (Shenzhen/Singapore) Odoo Gold Partner, best Odoo Partner 2014 for APAC
On 10/20/2015 02:38 PM, Raphaël Valyi wrote:
<blockquote cite="mid:CAByrsx2ooe=aQEni0u71tuEgpuSMw3fXdJ6tNJywn==yU-Z1iw@mail.gmail.com" type="cite">
I personally think we are loosing a bit the focus of the OCA here...

I think deployment will have many peculiarities depending on what want to achieve. Meanwhile a Dockerfile such as presented by Eric is really trivial to build and maintain (even accepting pull in his repo), I don't see any benefit of the OCA here... Even Voodoo which is a bit more engineered is trivial to maintain...

That is different for the Anybox recipe because it's still a large piece of software. And despite we may use the recipe differently, many of us will still use that part of the puzzle, so it makes sense to have it in the OCA (even if I was ok keeping Anybox in the name as I said).

Further, At Akretion we start using the Anybox recipe as a deliverable lingua franca. Like we happily host and make the infrastructure of our customers. But if they prefer do it themselves or work with other partners, we are OK as long as we can deliver them a buildout.cfg, so at least it means if they screw their production stack, that's not us paying their mistakes.

Finally, I think Docker and the recipe complete each other nicely and this is exactly what we do with Voodoo. Think that instead of using Virtualenv, we use a Dockerized virtualenv, but a voodoo project repo is totally deployable with virtualenv as usual for those who prefer. It's just we can get the project up on any computer in just 5 minutes without cluttering the developer small SSD disk with tons of native dependencies.

Regards.

On Tue, Oct 20, 2015 at 4:08 AM, Alexandre Fayolle <alexandre.fayolle@camptocamp.com> wrote:
I personnally see no issue with having both in OCA. I've been using buildout since long before docker existed, I depend on this recipe for a large number of customers, and this buildout recipe has gathered a significant community.

I am committing to participating in the maintenance it as long as I'm part of the association. I can't say the same for a docker based solution which is not part of the stack I deploy for my customers, but I won't block an initiative to maintain a docker based solution in the OCA.

Alexandre

2015-10-19 22:38 GMT+02:00 David Arnold <dar@devco.co>:
@Raphael, thanks! As I understand it, they do basically the very same thing?

from the anybox readme:

Some of its main features include:

  • uniformity across Odoo versions (from 8.0 onwards)
What does this mean, actually? Can't judge on this...
  • installation of Odoo server
Overlap: with docker capabilities.
  • retrieval of main software and addons from various sources, including the major version control systems
This is very nice, kind of "go get" or "vodoo get". 
  • ability to pinpoint everything for replayability
Overlay: Docker's main feature.
  • management of Odoo configuration
I imagine some advanced management of config files, this could be an interesting feature to spin out.
  • dedicated scripts creation for easy integration of external tools, such as test launchers
bash?!? is this docker's entrypoint.sh?
  • packaging: creation of self-contained equivalents for easy deployment in tightly controlled hosting environmenents.
Well nothing to say, this is dockers main purpose.

So basically we have 2 out of 7 main features that would high-lite in a "diff docker anybox_recipt". Couldn't those be spun out?

Obviously, things are working well, are stable and every one is accustomed to this. But I feel, my question if, on the long run(!), it is sustainable to maintain features inside the resource-limited odoo community, while other ecosystems solve them probably 100 times better (because they do nothing else).

I'm just thinking loud because everyone is complaining about not having the appropriate resources...

El lun., 19 oct. 2015 a las 12:53, Alexandre Fayolle (<alexandre.fayolle@camptocamp.com>) escribió:
Just for the record, I'm not suggesting we should make an exception in this case, and I certainly won't force an exception by abusing my commit rights and forcefully pushing the module in the OCA.

I've discussed things with Georges this afternoon and what I proposed him was:

* we change the name of the recipe
* we modify the original recipe by publishing a new version on pypi which will be empty save for a dependency on the new name of the recipe
* we add a compatibility entry point on the new recipe with the old name in order not to break existing buildout files

and globally he is ok with this.

I also feel such important decisions (accepting a whole new contribution etc.) deserve public discussions, so I launched the thread to tell the community about this and give everyone an opportunity to discuss.



2015-10-19 17:53 GMT+02:00 Raphaël Valyi <rvalyi@akretion.com>:
Hello Alexandre and others,

 I defended the idea that Odoo module should not have a company name because several companies would too easily work on the same topic and changing a module name has impact in the database and because it seems more important to have a name that reflects what a module does.

Now, for things like the Anybox recipe, I think Anybox will have done 90% of the work no matter the help now. It's not an Odoo module and I don't see reasons to be as strict. So for me I would leave the current name...

More generally, overall I like what we do with the OCA, but let me tell I'm concerned with the business model of the thing: yes the OCA receives some donations, but nonetheless 99% of the R&D is invested by the contributing integrators themselves.

May be some large integrators can do that as pure give away, but I'm not even sure. But generally, I think the model works IF AND ONLY F: an integrator company can get enough prestige from having its work accepted by peers at the OCA, then this can be a commercial advantage just as large as investing in marketing, at least in some rational high ends markets. THEN the model works and is sustainable.

Now, if the OCA ends up totally anonymising and hiding who did some work, if the OCA gives the same visibilty to somebody who created and carried a project 5 years as somebody that just helped over the last 6 moths, then my friends we are shooting ourselves in the feets big times... Because what we get then, is a some ugly lobbying to fake some project participation and hijack the project credits, and this is something already happening.,.

So I hope the OCA don't loose this economics reality. And this is for this reason I would be totally ok leaving the Anybox name in this case...

Regards.

On Mon, Oct 19, 2015 at 1:23 PM, Alexandre Fayolle <alexandre.fayolle@camptocamp.com> wrote:
Hello fellow,

I'm at the PyconFR sprint sitting next to Georges Racinet and a number of Anybox contributors.

Georges proposed a few months ago to share the maintenance of the anybox buildout recipe(s) (openerp and odoo) with the community. This was met with approval but as far as I remember the issue of the name (our policy says we should not have company names in module names) was not clearly resolved.


1. I propose the creation of a new OCA repository OCA/buildout_recipe under the responsibility of the Tools project steering comittee. I will make the required PRs to include the anybox recipes, once the naming issues are resolved

2. I'd like to propose Georges (https://github.com/gracinet) as a member of the tools PSC.

--
Alexandre Fayolle
Chef de Projet
Tel : + 33 (0)4 58 48 20 30

Camptocamp France SAS
Savoie Technolac, BP 352
73377 Le Bourget du Lac Cedex
http://www.camptocamp.com

_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe

--
Raphaël Valyi
Founder and consultant


_______________________________________________




--
Alexandre Fayolle
Chef de Projet
Tel : + 33 (0)4 58 48 20 30

Camptocamp France SAS
Savoie Technolac, BP 352
73377 Le Bourget du Lac Cedex
http://www.camptocamp.com

_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe

_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe




--
Alexandre Fayolle
Chef de Projet
Tel : + 33 (0)4 58 48 20 30

Camptocamp France SAS
Savoie Technolac, BP 352
73377 Le Bourget du Lac Cedex
http://www.camptocamp.com

_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe




--
Raphaël Valyi
Founder and consultant
+55 21 3942-2434


_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe


_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe




--
Raphaël Valyi
Founder and consultant
+55 21 3942-2434


_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe


_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe




--
Raphaël Valyi
Founder and consultant
+55 21 3942-2434


_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe


_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe




--

_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe


Reference