Contributors mailing list archives

contributors@odoo-community.org

Re: Recommended Contributor Tooling (RCT)

by
rvalyi@akretion.com
- 20/10/2015 19:19:02
Now guys,

speaking about about what can be universal or not to be at the OCA, may be we could standardize the base Buildout repo of a project?

So Voodoo uses that Buildout template:

As you can see, if you don't like Voodoo, you can clone a Voodoo project and run/deploy it with the standard Buildout tooling, with virtualenv or not.
And if you like Vodoo you just do
cd project
voodoo run
ak run
and your project is up and running on http://localhost:8069 (with a database that matches your project code tree)

Whas is opinionated in that Voodoo template is:
  • the dev buildout uses a file called buildout.dev.cfg
  • the project specifc modules live into a directory called "modules"
There is also a voodoo.yml which is a Docker Compose configuration file.

So in case may of you Anybox Recipe users like this template or want to suggest modifications to it, then I think such a template could join the OCA. In fact, for voodoo we would fork the OCA template and add voodoo.yml onto it or add it programmatic ally in the voodoo python launcher.

Benefit: if we all standardize on the same base template, with a project done for vendor X cloud technology will be trivial to make work on vendor Y cloud. So for instance say CampToCamp and Akretion can work on the same project, sharing the same git project repo, without having to change anything in their tooling and process...
Kind of a first step to tell our customers "the OCA guys can support your project"...

What do you think?

On Tue, Oct 20, 2015 at 5:00 PM, Raphaël Valyi <rvalyi@akretion.com> wrote:
@David,

and better than that, our embedded Postgres is optimized for speed, like with filesync=off
Something you will never do in production but that will sped up these awful cycles of --update=my_module cycles ...

On Tue, Oct 20, 2015 at 4:54 PM, Raphaël Valyi <rvalyi@akretion.com> wrote:
@David,

of course for production we use --link for Postgresql...

Now my friend, you cannot tell us you want a basic Dockerfile based installation a one click Windows experience and be against basic orquestration such as Docker Compose...

Look, when we talke about installation, postgresql and the Odoo settings of it is certainly one of the major hassle. On windows with windows users then...

The reason Voodoo comes with an embedded Postgres is that the developer don't even need to install Postgresql... As I said we wanted to make it as simple say as Rails projects using Sqlite for dev.

Now, despite we bundle Postgres, nothing of that is committed in the Project repo that remains a standard Buildout repo.
There is a .db file in the project (but in .gitignore), if you trash it you trash your dev database and nex run will create a fresh one.

The advantage is, you don't clutter a local Postgres server with gazillons of databases and then later loose time not remembering which project works with which database 3 months later etc..

But this is just the surface advantage. Our users can also pull the remote database to their local dev env in just one command. Our customers can have a local replica of their cloud instance they update in just a command (and soon via cron), then do some dev and push it to some our pre-prod server before we validate their dev and push it to the prod. But all that it compatible with the Voodoo Anybox recipe way of working, it's just not bundled with it and it remains flexible so cloud providers work the way they like and any git flow is supported.

Regards.

On Tue, Oct 20, 2015 at 4:38 PM, David Arnold <dar@devco.co> wrote:
Cool,

@ Raphael, I favor a postgres db linked with --link (possibly on another host, with easy tooling). This way you persist the postgres service independently from your efimeral dev environment. It is not following dockers philosophy at all to pack it into the python environment (for some very good reasons laid out in the docker manifests).

@ Big images: is not such a problem because they only use delta space to a common ancestor on disk. Read: when you see three images, 2 G each, that doesn't mean it need 6 G of disk space. So common tooling has some G's. psql does, ubuntu does, python does. So there is no way to avoid that entirely. But it is not a problem, nor is it something in which docker is worse than another solution in a direct comparison. The Contrary: docker excels through layered file systems and common ancestors. I don't go into the details, assuming that you are familiar with the basics of layered file systems. So I;m wondering about the whole point of the image-size discussion.
It's a question of optimizing a docker file and common ancestors. It's that simple than that...

Well I acknowledge, there is a point about additional tooling (such as devstep or others), which can cause unnecessary weight. If that is, what the discussion about size is about, I agree with Eric.

This is why I'm trying to extract voodoo functionality into a compiled binary. BTW, there can be anybox functionality comprised as well. The "go get" program (500 LOC) is the perfect blueprint. The rest is just a question, how to tune the perfect dev-docker file.

I don't think we should have different files here, but I think we should agree on a common set of best practices (about Dockerfiles). We should discuss technical details on each, and then take a decision. If for some tool compatibility (like with anybox, there is extra functionality needed, this special purpose dockerfile can just inherit from a common ancestor.

That's: easy, clean, collaborative, efficient.

Eric, Raphael, Moses, and others: What do you think of conforming a workgroup on defining the best and most modular/reusable dockerfile?

It can be in the form of a technical proposal (as github PR on maintainers-tools) so we can discuss and converge. I don't like the idea, that everyone is cooking in his own kitchen on this. Because such proceeding has it's inherent extra opportunity cost to all of us. Neither is it collaborative. 

What do you think? :D

Ps.: Why in the form of a PR? Because PR's let's easily comment very specifically and preserves entire history. It is very convenient to work on. (best practice from the docker ecosystem)

Best, David


El lun., 19 oct. 2015 a las 22:23, Eric Caudal (<eric.caudal@elico-corp.com>) escribió:

Hi Rafael,
Agree: this is a work in progress (there are some coming PR) but we want to keep it simple.
We want gather community effort around the subject to beef up the docker choice (same for Voodoo or anybox).

--
Eric Caudal [Founder and CEO]
Skype: elico.corp. Phone: + 86 186 2136 1670 (Cell), + 86 21 6211 8017/27/37 (Office)
Elico Shanghai (Shenzhen/Singapore) Odoo Gold Partner, best Odoo Partner 2014 for APAC
On 10/20/2015 10:07 AM, Raphaël Valyi wrote:
<blockquote cite="mid:CAByrsx2w2bO+LAzRvUn2kKk1G48-U69AOSe6-bqyJ9aRfhor0w@mail.gmail.com" type="cite">
Eric,

from a quick look what I see is your container is a bit minimalist while ours is opinionated toward the Anybox buildout recipe. And yes think there are audience for both. We will explain our choices about all that soon.

On Mon, Oct 19, 2015 at 11:53 PM, Eric Caudal <eric.caudal@elico-corp.com> wrote:

Thanks for feedback!
I understand the idea and advantages but it seems to me quite heavy for deployment actually

As I stated in another thread, I do think that there is space for all those tools anyway and that different audience will have different purposes.

--
Eric Caudal [Founder and CEO]
Skype: elico.corp. Phone: + 86 186 2136 1670 (Cell), + 86 21 6211 8017/27/37 (Office)
Elico Shanghai (Shenzhen/Singapore) Odoo Gold Partner, best Odoo Partner 2014 for APAC
On 10/20/2015 09:23 AM, Raphaël Valyi wrote:
<blockquote cite="mid:CAByrsx1h-S1_LNjKy6cWBmRzVffgAXecvkTuy_HHOJ9=Dm-=Aw@mail.gmail.com" type="cite">
Hello @Eric,

Well akretion/voodoo is based on Devstep as I explained and Devstep itself is 1.7 Go. That makes akretion/voodoo 1.93 Go large. Note that our akretion/odoocker prod image is much lighter than that by contrast.
We could optionally base voodoo directly from Heroku Cedarish, but we would loose many cool features... A great one is defintely the embedded Postgres that behaves like sqlite. Also at Akretion we often use Devstep for some other projects, specially Ruby where it avoids cluttering the developer machine with a Ruby+deps install, so in that context it even saves space to be based on Devstep. But Devstep is useful for many stacks, even Golang...

Also, as told Voodoo is compatible with Odoo 7, 8 and 9. So even if it had a 1 commit depth repo, that would be 3x the codebase, and also multiplied by 2x to support just OCB (but like on v7 we still have some rs-ocb branch to work around the partner model mess). So 6x the codebase, that would be heavy anyway... That's why we ended up leaving it outside from the imagine and up to the buildout recipe to download on some cached directory.


@David,

Well that's cool if you want to contribute a Golang launcher, but please don't change the name of the project, it really comes from the "Voodoo" English word and it's a kind of therapy for OCA community enabling them to imagine doing some Voodoo with some Odoo developers after they dare to packaged it with so many bugs inside :-p


Finally, Akretion will soon participate to some code-sprint and demo Voodoo. We will also soon blog about it and demo it. Hope you enjoy it as much as we do.

On Mon, Oct 19, 2015 at 10:08 PM, Eric Caudal <eric.caudal@elico-corp.com> wrote:

Hi Rafael,
Our docker images are 1Gb heavy (only one commit deep): Any reason why such a difference?
Best regards

--
Eric Caudal [Founder and CEO]
Skype: elico.corp. Phone: + 86 186 2136 1670 (Cell), + 86 21 6211 8017/27/37 (Office)
Elico Shanghai (Shenzhen/Singapore) Odoo Gold Partner, best Odoo Partner 2014 for APAC
On 10/20/2015 03:22 AM, Raphaël Valyi wrote:
<blockquote cite="mid:CAByrsx2Zeb2S0ven9RgcMjC_davJXQoFKT=GGUth8mhbw7y4sA@mail.gmail.com" type="cite">
Moises,

I'll look at what you did, and probably there s some convergence. I see value in being able to spin a Dockerized Runbot in anyway...
But about the cache specifically; In Voodoo we wanted to make it work with both Odoo 7, 8 and 9, official Odoo branches, OCB or even custom fork. If we putted all that in the image, that would make the image more than 1Go more to download and then unpack on the disk on our poor SSD disks... (The Voodoo image is already 2Go, making it 3Go would kind of kill the user experience) So instead we let the buildout download a shallow clone of the wanted Odoo repo and we cache it somewhere on the host. I'm open to change that logic but this is how we did until now. And this is more or less the same logic for the Python eggs.

Regards.

On Mon, Oct 19, 2015 at 3:08 PM, Moises Lopez <moylop260@vauxoo.com> wrote:
> basicallly what sucks in Odoo is the the repo is huge. So a large part of the Voodoo Python launcher is made to create the options of the docker run command, specially which folder you'll mount to share your Odoo repo (Odoo is too much of a buggy moving target to be bundled inside the image and it's to heavy to be downloaded always).

Yes this is a big problem because the creation of instance is too slow.
FYI We fix it with a "cache" into image similar to travis build cache.
We have in base image a `git clone odoo` then after create a container MQT execute a `git pull odoo VERSION`
Now you don't need download each `docker run` full odoo repo just download last changes from last build image.

I will review the Consul process to know other way.
Thnk you.


2015-10-18 20:07 GMT-05:00 Raphaël Valyi <rvalyi@akretion.com>:
David,

basicallly what sucks in Odoo is the the repo is huge. So a large part of the Voodoo Python launcher is made to create the options of the docker run command, specially which folder you'll mount to share your Odoo repo (Odoo is too much of a buggy moving target to be bundled inside the image and it's to heavy to be downloaded always).

Alternatively I thought about putting the Voodoo inteligence inside the portable Linux image and like running it twice: one to first output the run options and then eval the command of the second run. That's e technique used by Consul https://github.com/gliderlabs/docker-consul/tree/legacy#runner-command but again sadly that wouldn't work on Windows.

Voodoo also have a few other useful commands. We often use "voodoo new project_name" to bootstrap a new project and hack on something.

Overall, I think the Voodoo Python launcher is useful. But what I think is we can have 80% of the useful features in just 20 lines of Ruby in a Vagrantfile. Then "voodoo run" would be just "Vagrant up" on windows and sharing a project would just work (not the advanced Docker compose feature but f*ck Windows right...)

So again, pull request for a such a Vagrantfile would be greatly appreciated in Voodoo.

Regards.

On Sun, Oct 18, 2015 at 10:38 PM, David Arnold <dar@devco.co> wrote:
Hi Raphael, I'm right now analyzing in detail the docker onboarding guide, that's just a bliss. I'll rewrite it for OCA. THE thing is: We can probably completely drop docker-compose and just work with the priviledged flag, to run multiple containers within the first. thise moves tooling environment definition out of the user machine and into a second layer easily managable via a DOCKERFILE... I'll keep you posted, if that might work...

El dom., 18 oct. 2015 a las 15:07, David Arnold (<dar@devco.co>) escribió:
Thank you nhomar for your opinion.

It is indeed a valuable point, but it is not breaking, because:
a) Easier is always better then complicated.
b) I'm willing to take on the challenge. So I hope, once testable the scope might be reconsidered upon the final thing.

In the end, we have multiple error layers when someone want's to get started on hacking openobject framework, among them are:
virtualisation, autoreloading, debugging, editor, templating, prototyping, python environment, os environment (external dependencies). Some of them take weeks to understand and solve, and for some, there is such a flood of possibilities that chances are high you just give up on the way. This means valuable brainpower lost. (It doesn't serve us someone finding out how to setup his stack, it only serves us someone contributing openobject code, so let's get them there in no time! And this marginal contribution can possibly be the so hardly needed reviewing activity...)

I think this is why broadening the scope, once we have a viable solution is worthwhile..

Guys, share me some enthusiasm! :) That could be more fun for everyone...

Best, David

El dom., 18 oct. 2015 a las 14:52, Nhomar Hernandez (<nhomar@vauxoo.com>) escribió:

On Sun, Oct 18, 2015 at 2:38 PM, David Arnold <dar@devco.co> wrote:
Sorry, you just don't want to burden this onto someone who just want's to get started. Even to us it might be an obvious error. It's not the user's fault, that this is not working. It is the programmer's (as almost always) ...

I insist... you are trying to solve an issue far away from the objective of odoo's project.

If somebody want install odoo and is new and do not understand that error, need to google 2 or 3 minutes and will find the solution.

Even it is better if you recommend 2 or three reads before start.

     odoo developer book

If you buy i.e Mac or Windows apps.... the next next buttons will make you some questions which th documentation explains conceptually what they are, but if the user do not have tha minimal knowledg you can not expect the system will solve a connection to their brains to teach alá matrix!

Regards.

--
Nhomar Hernandez
CEO Vauxoo.
Twitter: @nhomar
Odoo Gold Partner
Móvil Venezuela:
+58 4144110269
Móvil México:
+52 1 4773933942

_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe

_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe




--
Raphaël Valyi
Founder and consultant


_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe




--
Moisés López Calderón
Vauxoo - OpenERP's Gold Partner
Mobile: (+521) 477-752-22-30
Office: (+52) 477-773-33-46
web: http://www.vauxoo.com
twitter: @vauxoo
           @moylop260           

_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe




--
Raphaël Valyi
Founder and consultant
+55 21 3942-2434


_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe


_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe




--
Raphaël Valyi
Founder and consultant
+55 21 3942-2434


_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe


_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe




--
Raphaël Valyi
Founder and consultant
+55 21 3942-2434


_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe


_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe

_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe




--



--



--
Raphaël Valyi
Founder and consultant
+55 21 3942-2434