1. Mailing Lists
  2. Contributors
  3. Re: 30 Checkout Operations per Minute A job for Odoo or not ?

Contributors mailing list archives

contributors@odoo-community.org

Browse archives

Re: 30 Checkout Operations per Minute A job for Odoo or not ?

Re: 30 Checkout Operations per Minute A job for Odoo or not ?

Avatar

Re: 30 Checkout Operations per Minute A job for Odoo or not ?

by
Vauxoo, Nhomar Hernández
- 10/05/2021 19:57:12


El jue, 6 de may. de 2021 a la(s) 05:02, Frederik Kramer (frederik.kramer@initos.com) escribió:
Hi Moises, 

Am Freitag, den 30.04.2021, 16:02 +0000 schrieb Moises Lopez:


> Hi Frederik


> 


> A customer requested to support 5k concurrent sale.order (1 or 2


> lines of products) in Odoo.


> The process that we followed were: test, measure, fix, test...

haha, yes thats what works well if you have a customer that well
endorses this "agile" and "pivoting" style of work. Apart from all
economic considerations this is probably also something that isolated
leads to quite solid solutions

That was our biggest measured customer (a recovered project) they know their KPI very well, on simpler customers, we try from a commercial perspective to explain exactly what Moises does (in a more extensive/commercial way) to try to teach them how to buy technology.

At the end if you try let's say Shopify (the king of marketed e-commerce) with those numbers you mentioned or this ones they will offer you more **Enterprise** services (note the services not the software) because almost everybody works on this way even the order on which your usual customers make click on the website can change the outcome and bring performance/consistency problems, i.e: click twice a button in an slow process that by mistake land an specific day on your code).

Then what's important IMHO is to clarify to the customer that set technically solid e-commerce is 30% software product 50% the right team.

If this was not the case, we would have millions of amazon stores, Don't you think?


> 


> The first time that we tested the checkout process supported 15


> concurrent virtual users (VUs)


> The measure said that it had heavy custom functions in the same


> transaction of checkout but it could be executed in background (cron,


> queue)


> and there were slow queries because of indexes issues or too many


> queries executed because of python code

did you use any specific profiling tools here? Or the ones that rcollet
often proposes in his talks about the matter ?
Be careful here, they are a lot, and in my honest experience that's more the person using the tool than the tool itself.


> 


> Note: Be careful with the base_automation if you can migrate them to 


> code it is better.

Thanks for the hint, i think we experienced that also quite some times
in the past and learned that lesson already ;-)



> 


> After fix them the next iteration supported 400 VUs


> So good but so far from the focus.


> 


> The next iteration we found that there were methods that could be


> using LRU-cache (e.g. prices, website that don't change)


> Now we needed to check where the cache was clearing too frequently


> and fix it.

How did you measure LRU cache hit rates vs. code parts in a good way.
Any tooling suggestion here ?

1. log-sql and a strong test setup that emulates the production environment is required.
2. We use some SQL analyzer tools as well.

But again, it is a matter of having the right team doing the job because it is not theoretical work, it is practice, test, read, practice, learn, fix, start again. and a little bit of good luck on some cases and being stubborn to solve the issues ;-)


> 


> We implemented CDN


> e.g. The sentence "/web/image/company.logo" runs 10 queries


> (prefetching fields, translating fields...) for each user opening the


> page


> After using CDN 0 queries are executed.

But you just used Odoos standard CDN function or any specific measure
here ?

Standard! we simply use Odoo's CDN functionality.


> 


> We removed soft process that running with many users it is heavy


> e.g. the random image that odoo assign when you create a res.partner


> e.g. computed fields moved to cron

Also here which analysis tools do you prefer to do so ?



> 


> After that we detected concurrent update issues


> It is hell for performance matter.


> e.g. updating the same record at the same time from many users,


> e.g. allowing pressing the same button too many times


> e.g. sequences using no-gap in the main process, and so on


> 


> After finishing all the hints of performance issues in the code and


> database the next steps were to improve the hardware.


> 


> Now, It supports 5k concurrent VUs doing checkout process (choose 2


> products, type address, payment card info and confirm)

In what time frame ?

With the real use case and in a production environment giving the problems more or less 60 days between First work and release into production ready for the action.


> 


> So, Odoo will support those users but you will need to test, measure,


> fix, test...


> Stress testing tools helps us a lot here


> python profilers and query analyzers too


> 


> Maybe you are interest to watch our talk in SFO about:


>  - https://youtu.be/kiREZqg3JQg

I will definitely have a look at this

Thanks a lot for your extensive insight.
I summarize with:

The team is the key to success here, every single piece in any e-commerce on this use case was given by different persons and led by Moisés because problems are in all layers: BD, Server, Sizing, Odoo, JS, OCA.... 

It is not just one single pill!
Cheers from rainy Germany to sunny Mexico ;-)

Frederik



> 


> El mié, 28 abr 2021 a las 15:35, Frederik Kramer (<


> frederik.kramer@initos.com>) escribió:


> > Hi Folks, 


> > 


> > i have a project proposal on my desk that will ultimately end up in


> > approx. 30 checkout transactions per minute. Has anybody got a


> > project


> > with that order volume done with a relatively recent version of


> > Odoo


> > and provide a good reference. Please bear in mind that the project


> > ahead involves a similarly high volume of stock moves.


> > 


> > Any suggestion / comment like (but not limited to) 


> > 


> > Odoo is the wrong technology for that, forget it, because....


> > 


> > over


> > 


> > You can do it but, a), b), c) ....


> > 


> > to


> > 


> > If you really want that beast flying involve us over here because


> > we


> > are among the few ones that already did soemthing similar


> > 


> > would be appreciated. 


> > 


> > From our load intensive projects at initOS i know that 10000 orders


> > a


> > day can be handled but almost 50000 is a different story and thats


> > why


> > i am asking the most experienced resource out there ;-)


> > 


> > Thanks in advance for your replay


> > 


> > Cheers Frederik


> > 


> > 


> > -- 


> > Dr.-Ing. Frederik Kramer


> > Geschäftsführer


> >         


> > initOS GmbH


> > An der Eisenbahn 1


> > 21224 Rosengarten


> >         


> > Phone:  +49 4105 56156-12


> > Fax:    +49 4105 56156-10


> > Mobil:  +49 179 3901819


> >         


> > Email: frederik.kramer@initos.com


> > Web:   www.initos.com


> >         


> > Geschäftsführung:


> > Dr.-Ing. Frederik Kramer & Dipl.-Ing. (FH) Torsten Francke


> > 


> > Sitz der Gesellschaft: Rosengarten – Klecken


> > Amtsgericht Tostedt, HRB 205226


> > Steuer-Nr: 15/200/53247


> > USt-IdNr.: DE815580155


> > 


> > _______________________________________________


> > Mailing-List: https://odoo-community.org/groups/contributors-15


> > Post to: mailto:contributors@odoo-community.org


> > Unsubscribe: https://odoo-community.org/groups?unsubscribe


> 


> 


-- 
Dr.-Ing. Frederik Kramer
Geschäftsführer
        
initOS GmbH
An der Eisenbahn 1
21224 Rosengarten
        
Phone:  +49 4105 56156-12
Fax:    +49 4105 56156-10
Mobil:  +49 179 3901819
        
Email: frederik.kramer@initos.com
Web:   www.initos.com
        
Geschäftsführung:
Dr.-Ing. Frederik Kramer & Dipl.-Ing. (FH) Torsten Francke

Sitz der Gesellschaft: Rosengarten – Klecken
Amtsgericht Tostedt, HRB 205226
Steuer-Nr: 15/200/53247
USt-IdNr.: DE815580155

_______________________________________________
Mailing-List: https://odoo-community.org/groups/contributors-15
Post to: mailto:contributors@odoo-community.org
Unsubscribe: https://odoo-community.org/groups?unsubscribe



--

Nhomar G Hernández

Vauxoo | CEO

¡Construyamos algo genial!
Cel: +52 (477) 393.3942 | Telegram: nhomar | Twitter: @nhomar

México · Venezuela · Costa Rica · Perú

phone nhomar@vauxoo.com phone vauxoo.com/contactus  

Reference