Contributors mailing list archives

Hello David,

some answers:

1. I would be open to having voodoo installed inside the Docker image and launch it via "inception". Now, I'mo not sure this is easy. One thing is that volumes shared with Docker (via -v) should be absolute paths, not that easy to achieve in a portable way on both Windows and Linux (you should share you project workspace and the shared odoo local repo at least).
2. I'm totally open if you want to rewrite Voodoo in golang or hack into the Devstep launcher enough to get it able to Launch the Voodoo image with the proper options. It's just that everubody uses Ubuntu at Akretion and we know Python so it was just about getting fast to the point.
3. In any case Docker compose is absolutely useful (but we could accept not having it in a degraded windows mode). It's really a design shame that Odoo is such a monolithic design, specially now that CMS crap. At Akretion we do several project with a Ruby web part or even Apache Solr and sharing them with Voodoo is just so much easier. Docker opens up a whole new world of micro-services. Launching a configured Eclipse+PyDEV or a Database browser are quite useful for the noobs.
4. We deploy to the prod via git push ("git deploy"), we have special git hooks on the other side of the network. Now Odoo hosting in production is a whole subject in itself with many possiblities (shared host, isolated containers, auto-scaling cloud etc...) and is out of the scope of Voodoo which aims to be just a development tool. I suggest you contact me privately if you wish to collaborate seriously on these other topics.

Regards.

On Sun, Oct 18, 2015 at 11:23 PM, David Arnold <dar@devco.co> wrote:

Raphael, thanks a lot for your detailed explication!

This gives me the possibility to ask some more detailed questions, I'm interested in:

Voodoo is basically a pre-built dev environment bundled a single Docker image. We think it's the right mileage between the pre-build-dependencies for a turn key startup and the flexibility of Anybox recipe to deal with any module branch and Odoo branch, just like using the recipe without Docker. 

I wanted to open up discussion on this point because of the following reason: travisfile2dockerfile by vauxoo permits using the travisfiles, that since the implementation of travis by oca are present (!) and maintained (!) in every repo. travisfile2dockerfile adds therefor a dynamic environmentcreation, which is the same goal what you want to achieve with anybox, if I understadn you right. On the other hand, and this is the elegance I was aiming for, Moses probably can easily rewrite his t2d library to work as a maintenance script, that automatically transforms and updates (via PR and Github Bot Account) each repo's individual "last mile dockerfile" (based on a common oca base image). This way, I can easily implement and transform the docker contribution guide and adapt it to the oca ecosystem. The only host dependency is indeed git and docker, this way. (and docker-machine abstraction handler on OSX and Windows).

It makes projects very lightweight, self-contained (even the dev database is inside the project folder, just like Rails with sqlite) and extremely easy to share between developers. And we also have easy tools to push such a dev env to the cloud in just one command.

I assume this command is the generic docker export / docker import or some lightweight scripting and scp streaming arround it? Or can be acomplished that way? As I understand you, you simply persist the database volume onto the host for direct acces? This might be a good and desirable idea, and is supposedly accomplished by docker -v command, right? I take note of that, to include it into the contribution guideline, I'm aiming at. 

The python package of Voodoo is just a handy launcher. initially it was 20 lines of bash but then my co-workers rewrote it as a Python wrapper around Docker Compose, giving it unlimited flexibility to attach custom micro-services such as a mail catcher, a pre-confgured Eclipse, a database browser, a Solr server, a Rails e-commerce, anything you want, specified in the same project repo.

That is a very noble intention. I would however point out, that it is important to not depend on any local host dependency. But that wouldn't probably be a problem, you can easily rewrite that and include it into the docker host itself. This is called docker inception technique. It would feel something like docker run --privileged aktretion/voodoo commandxyz. Neither bash nor python are good enough for the local host, because of portability. But I feel we do not need any tooling, one git and one docker command is completely enough (just 2 standard BINARY commands) To setup the perfect dev environment. Can those microservices not be achieved by a standard docker-compose definition? We could move the "necesary" to a docker in docker "privileged" architecture, such as db, debugging and the odoo environment and leave optional microservices to an still optional docker-compose (like this windows user are not doomed).

Now yes, this depending on the maturity level of Docker compose on Windows and Mac, the launcher may not work on these platforms. Well in that case, in a few hours of work we can ressucitate the Vagrantfile launcher that will mimic the 80% of important features to fire the Docker workspace container on these plateforms.

I'm actually against this approach, because it add's even more unecessary plumbing and tooling to the local host. I'm very against this, because it increases setup times. Actually I have in my mind a design, where we would not need docker-compose, so this would be elegantly resolved by "reduce to the max" strategy.

I believe this is what you wanted to achieve. And yes it's mature, we have been more than a dozen of people using it for 6 moths now at Akretion...

That is definitely a valuable argument, however, if we could reach out to completely cut out this layer at all, which needs stabilization, why shouldn't we?

Best Regards, David

El dom., 18 oct. 2015 a las 19:38, Raphaël Valyi (<rvalyi@akretion.com>) escribió:

David,

Voodoo is basically a pre-built dev environment bundled a single Docker image. We think it's the right mileage between the pre-build-dependencies for a turn key startup and the flexibility of Anybox recipe to deal with any module branch and Odoo branch, just like using the recipe without Docker. It makes projects very lightweight, self-contained (even the dev database is inside the project folder, just like Rails with sqlite) and extremely easy to share between developers. And we also have easy tools to push such a dev env to the cloud in just one command.

The python package of Voodoo is just a handy launcher. initially it was 20 lines of bash but then my co-workers rewrote it as a Python wrapper around Docker Compose, giving it unlimited flexibility to attach custom micro-services such as a mail catcher, a pre-confgured Eclipse, a database browser, a Solr server, a Rails e-commerce, anything you want, specified in the same project repo.

Now yes, this depending on the maturity level of Docker compose on Windows and Mac, the launcher may not work on these platforms. Well in that case, in a few hours of work we can ressucitate the Vagrantfile launcher that will mimic the 80% of important features to fire the Docker workspace container on these plateforms.

Imagine that Voodoo is just Devstep for Odoo:

http://fgrehm.viewdocs.io/devstep/

I believe this is what you wanted to achieve. And yes it's mature, we have been more than a dozen of people using it for 6 moths now at Akretion...

Cheers.

On Sun, Oct 18, 2015 at 5:38 PM, David Arnold <dar@devco.co> wrote:

Example, according to documentation on an XYZ python library:

$ python setup.py develop

Traceback (most recent call last):

  File "setup.py", line 11, in <module>

    from setuptools import find_packages

ImportError: No module named setuptools

Sorry, you just don't want to burden this onto someone who just want's to get started. Even to us it might be an obvious error. It's not the user's fault, that this is not working. It is the programmer's (as almost always) ...

El dom., 18 oct. 2015 a las 14:24, David Arnold (<dar@devco.co>) escribió:

Ps.: If you can ship me a windows 64 compiled binary, I'm happy to evaluate it thoroughly!

El dom., 18 oct. 2015 a las 14:22, David Arnold (<dar@devco.co>) escribió:

Hi Raphael,

thanks for your feedback. Indeed it is possible. Actually I'm still waiting more detailed technical description of voodoo to be able to draw my own judgment. A statement  "will be exactly what you want" assumes that you "understood exactly what I want". What I saw from a quick look it is a python library. So unfortunately this is not a common enough ground for tooling, mainly because it leaves higher level tooling (like python and virtualenv) to the user. This is not nice and not even necessary. Please correct my, though, if I understood the functioning of voodoo wrong. Again, I would be delighted by a short technical description comprising at least a description of what issues are addressed by what strategy. That would be really helpful.

Thanks and best regards, David

El dom., 18 oct. 2015 a las 13:08, Raphaël Valyi (<rvalyi@akretion.com>) escribió:

David,

sorry but I really think if you help contributing a Vagrantfile launcher for Voodoo it will be exactly what you want and following best field practices...

Without help we will make that Vagrantfile (unless Docker compose run smooth on Windows) but help could speed this up.

Regards.

On Sun, Oct 18, 2015 at 5:23 AM, David Arnold <dar@devco.co> wrote:

Moses, we have the problem, that most windows user, that install git via he official installer will opt for the (recommended) option that files are checked out with CRLF. I have the gut feeling, that if they have done so, bindmounting those files will produce non understandable errors in the outside layers of their stack for them. This would be another argument for having the git proxy and working over a network on the docker filesystem. Or we populate respective sources into bindmounted directories AFTER the startup in the run script... I mean to be able to leverage docker volume and OS standard tooling instead of having to especially prepare the IDE. And keep that stuff out of the odoo environment.

El dom., 18 oct. 2015 a las 1:47, David Arnold (<dar@devco.co>) escribió:

Hi Eric, thank you for your feedback!

What I want to achieve is a standard way for new contributors to get coding quickly, weather they be on windows, osx or linux. My role model newcomer does not know python, nor git, nor does he know other stuff. He knows barely some basic concepts of the command line, and still he is valuable, because he might be able to copy paste some code and get his business case working. And, he might grow and evolve...

Installing python is cumbersome and litters your computer, and after you do a computer reset (I do that every 2 months probably) you have to start again setting everything up. Setup virtualenv is even more cumbersome, actually I never got a hold on it. So be python based solution discarted. We need binaries to get people going. Enter docker, it ships binaries. But I'm not getting at deploying. I'm aiming at the perfect, most cross platform, easy, dead simple, get going in less than 5 seconds development stack.

The point in the above mail is derived from Moses's (i feel) briliant idea to reuse the maintainance work, that is already present in the travis files. He wrote a tool, that transforms it into a buildable docker file (https://github.com/vauxoo/travis2docker). This set's up a complete environment (python deps, odoo deps other deps). So if you want to start hacking on some oca repo, you just have to clone it and build the automatically maintained dockerfile which includes the specific environment of this specific repo. And maintainers would carry on maintaining the travis files only, which have kind of the historic precedence rights on their side.

After building and docker run, you can hapily modify the src on your host and everthing get's bindmounted or synced into the docker, so you go to localhost:8069 and see what you have done instantly (with pynitify and odoo autoreload).

That's the kind of instantaneousness I want to see, also on windows systems. And I have not mingled one single time with anything of the kind of virtualenv, pythondeps, git blabla, missed module dependencies. It's just something like git clone && docker build . && docker run. Required binaries: git, docker. That's all - available for nearly any thinkable of common platform. And you are good to go hacking on your working directory.

Not: This is quite agnostic of whatever (probably git based) upper continuous integration workflow you chose. But this is anyhow not a concern of the one how just wants to get his hands dirty.

I'm trying to collaboratively discuss the best design for such a tool and am committed to come up with a beta-worthy solution for testing. Maybe I need some help here and there, but with communities support, I think it's doable.

Best, David

El sáb., 17 oct. 2015 a las 22:08, Eric Caudal (<eric.caudal@elico-corp.com>) escribió:

Hi David,
We appreciate your  various contribution but in this case, I am not sure to understand where you are getting at.
Docker is very nice environment and I am in favor for the deployment although the techno is style young is not completely bullet proof.
We have our own here based on OCB and refreshed  every week (as of next week) : https://hub.docker.com/u/elicocorp/.
You can grab docker-files here:https://github.com/Elico-Corp/odoo-docker
(I will send a separate mail anyway to check if community could be interested in maintaining this repo in OCA)

But I dont see the point of your exercise. Current infrastructure is solid for travis: once your code is solid and your docker file is well structured, there is no need to travis the dockers (IMHO).

Resources are limited and we do not want to tumble our infrastructure that we have needed month to stabilize so we need clear objective and pros for the community and you will come with a fully prepared solution.
We barely have resources for maintainance and cannot really just TRY things...
If you can come with solid project definition and proof of concept + some resources to maintain it in the community, we could consider it

--
Eric Caudal [Founder and CEO]
Skype: elico.corp. Phone: + 86 186 2136 1670 (Cell), + 86 21 6211 8017/27/37 (Office)
Elico Shanghai (Shenzhen/Singapore) Odoo Gold Partner, best Odoo Partner 2014 for APAC

On 10/18/2015 10:37 AM, David Arnold wrote:

<blockquote cite="mid:CAOLEt-EMOGXd9JjKassUDD6zkzOD5ZFfsgcop13qFMD1TpJM9Q@mail.gmail.com" type="cite">

Having talked with Moses (vauxoo) bilateral about his solution further, I would like to gather support for the following attempt (ATTEMPT = TEST = TRY = EXPERIMENT): 

- Transform travis to use docker

- Run travis2docker (or a small modification of it) to create DEV docker files in repo

- Start a dev environment with docker build . && docker run and easily mount the working tree into it's place

Best advantage: Binaries for almost every platform are available in the docker ecosystem.

Manual maintenance can focus on the Travis files, the rest is already scripted...

@Moses: when talking about mounting instead of syncing via a network protocol via the docker volume command (read: let the docker host take care of this) we cannot AFAIK let the docker be the source, because docker gives precedence to the host files system and eventually deletes anything that was present in this path on the docker itself. So we might have a closer look on how we could include the git proxy workflow nevertheless. It's important though, that a local git can be easily referenced, so no internet connection is needed...

Thanks for your questions, if any!

Best, David

El vie., 16 oct. 2015 a las 11:58, David Arnold (<dar@devco.co>) escribió:

Let's turn to https://github.com/OCA/maintainer-tools/issues/146 for the tooling discussion, anyone who can moderate or better elaborate the facts, pros and cons, than I'm probably capable of: please be welcome!

Inputs from akretion and vauxoo are very promising, we need to decide on the right dosis of tooling for this specific purpose, and assure plattfrom independence.

El vie., 16 oct. 2015 a las 11:52, David Arnold (<dar@devco.co>) escribió:

Adapted.

El vie., 16 oct. 2015 a las 11:23, Daniel Reis (<dgreis@sapo.pt>) escribió:

I just took a closer look at it.
Words like "Recommended Editor" are notorious for triggering epic flame wars, such as emacs vs vim ;-)

Às 16:53 de 16/10/2015, David Arnold escreveu:

<blockquote cite="mid:CAOLEt-E1f1QPZht+0db==8EbPAxfX_4vyz7uTQxp1Po800UhEg@mail.gmail.com" type="cite">

Hi!

Based off on https://github.com/OCA/maintainer-tools/issues/146 and Daniel's suggestion, I made a recompilation here of recommended contributor tooling: https://github.com/OCA/maintainer-tools/wiki/Recommended-Contributor-Toolbox-(RCT)

It's WIP, but with your feedback, I'm commited to make it 1.0 at the indicated detail level in no time. I hope it might also trigger or revive connected discussions, such as repo layout. (https://github.com/OCA/maintainer-tools/issues/147) - For my taste, I invite you to discuss directly on the github issue to not bother the less involved.

I'll reference also a second mail to follow, with a proposal for modernizing informal communication with the help of gitter.im.

Best Regards

David

_______________________________________________

Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe

_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe

_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe

_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe

_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe

--

Raphaël Valyi

Founder and consultant

http://twitter.com/rvalyi

+55 21 3942-2434

www.akretion.com

_______________________________________________

Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe

_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe

--

Raphaël Valyi

Founder and consultant

http://twitter.com/rvalyi

+55 21 3942-2434

www.akretion.com

_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe

_______________________________________________
Mailing-List: http://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: http://odoo-community.org/groups?unsubscribe

--

Raphaël Valyi

Founder and consultant

http://twitter.com/rvalyi

+55 21 3942-2434

www.akretion.com