Duration 6:5

Top 7 Ways to 10x Your API Performance

253 992 watched
0
11 K
Published 27 Jul 2023

Get a Free System Design PDF with 158 pages by subscribing to our weekly newsletter: https://bytebytego.ck.page/subscribe Animation tools: Adobe Illustrator and After Effects. Checkout our bestselling System Design Interview books: Volume 1: https://amzn.to/3Ou7gkd Volume 2: https://amzn.to/3HqGozy The digital version of System Design Interview books: https://bit.ly/3mlDSk9 ABOUT US: Covering topics and trends in large-scale system design, from the authors of the best-selling System Design Interview series.

Category

Show more

Comments - 156
  • @
    @wissemaljazairi9 months ago 1. Caching
    2. Connection Pool
    3. Avoid N+1 Query pattern
    4. Pagination
    5. JSON Serialization
    6. Compression
    7. Asynchronous logging
    ..
    ...Expand
    74
  • @
    @bananesalee70869 months ago For some reasons, listening to you is calming. 17
  • @
    @narasimhareddy83239 months ago One quick question.
    who does the video animation work for you? Kudos to the designer whoever he/she is.
    24
  • @
    @gregoirehebert9 months ago Around the n+1 problem, i think it' s worth mensionning that using http cache on the comments would reduce the amount of processing. No need to claim but you still need to request them at some point, even if this is to the cache reverse proxy.
    to avoid waiting your frontend to parse the payload then query the comments, the use of 103 earlyhint can eliminate that waiting time.
    using http/2 as it is using binary frames, multiplexing and solves hol blocking.
    using http/3 as it speed up establishing the connecion api.
    formats like protobuff also reduces the size of the payload.
    to circumvent pipelining problems using http/1. 1, sometimes batch http request can be a solution (prefer standard specification) but please stay stateless as much as possible.
    of course the infamous domain sharding
    .
    ...Expand
    11
  • @
    @asadbekfayzullaev9 months ago Thank you! I love watching bytebytego system design videos! 5
  • @
    @kns61329 months ago Superbly explained and very valid tips. 12
  • @
    @guhkunpatata31508 months ago The animation + explanation is great! Thanks for sharing!
  • @
    @harshdevsingh65069 months ago 1. Tunning database tables like purging old data which is not needed can minimize the performance of select queries on tables.
    2. Putting the index 3. Adding load balance also helps to improve the performance.
    4. Payload compression also can be helpful to fetch large size data like image/videos
    .
    ...Expand
    5
  • @
    @Ajdin879 months ago I think it is worth mentioning that if you decide to use caching, redis, and you are using, for example aws, it will be additional cost. Caching is done great video btw. ...Expand 19
  • @
    @MrSongsword9 months ago The 7 methods:
    1. Caching
    2. Connection pool
    3. Avoid 4. Pagination
    5. Json serializers
    6. Payload compression
    7. Asynchronous logging
    .
    ...Expand
    46
  • @
    @anthonysalls9 months ago An architecture i work with involves a secondary copy mirror, and i once crashed the mirror by supplying too many writes to primary that were handled eff switching to asynchronous log writing dramatically improved the performance and the secondary system could handle the load fed from primary again, but there was half a day of 40, 0 users who were not happy that their reports when from taking half a minute to run to half a day. Additionally ha and data loss was risked as the secondary system was the same-data center backup of primary (there was another primary off site with similar hardware that kept pace but would have taken longer to fail over to)
    the lesson is, if production is reliant on secondary systems it communicates with, and youre going to be running production hot for hours, you must have secondary systems attached to your test suite! Wed 4 dozen attached but missed the mirror that was instrumental for reporting
    .
    ...Expand
    4
  • @
    @makhaer9 months ago @bytebytego could you tell which tool you use to draw these animated diagrams? 2
  • @
    @nmmm20009 months ago 2 cents from me too:
    - use http keep alive or http2 - if you do separate http call for each api call, speed will be slow. If you do http keep alive, - optimize sql queries:
    - optimize sql queries:
    - optimize sql queries:
    - database replication
    .
    ...Expand
    36
  • @
    @MichaelKubler-kublermdk9 months ago Something we recently did was use amazon sqs to push after save and after update tasks to a background processing server.
    this allows things like of course we are using php not something like nodejs where processing after an api response is much easier. ...Expand
    3
  • @
    @thanhnx-vnit9 months ago @bytebytego
    video is great. Thank you so much!
    btw, could you tell me the software to create presentation slides like in the video?
    2
  • @
    @leomysky9 months ago Wonderful
    thank you for the video.
  • @
    @ravipvkiranpv9 months ago Add streaming to the list. Great video.
  • @
    @jacob_90s8 months ago One issue to look out for with pagination when using top/max and offset are changes that occur between page request and how they affect the order of the . ...Expand 6
  • @
    @arfinexe5399 months ago Great video, but that car transition caught me off guard. 3
  • @
    @khari_baat9 months ago Which efficient json serializer are we talking about? Can you please suggest some? 1
  • @
    @pauljohnsonbringbackdislik14697 months ago Since browsers limit active parallel downloads, there are some cases when batching requests may shave 2-5s from the overall page load (e. G. On pages that i wish @bytebytego could make a follow-up video with " top 7 ways to optimize performance suggested by:. ...Expand
  • @
    @leetcode78579 months ago Other techniques:
    1. Tuning the database connection pool size based on the application behaviour. Large number doesn' t always mean more performance) 2. Optimizing the sql query. Ensuring your most frequent queries end up using index scan instead of full table scan)
    3. Not hopping between multiple microservices for a single user request. While a single user request can hit multiple services but those services should not in turn hit another set of services and so on)
    4. Authorization data should be always cached.
    5. As much as possible, do the most computation on the database layer. There' s a huge difference between doing the computation at application layer vs doing it at database layer
    .
    ...Expand
    260
  • @
    @dmitrypichugin74499 months ago Bulk rest endpoints (controller architype)
    rest -gt; graphql.
    2
  • @
    @nagsworld8 months ago Your video' s looks greate, mainly presntation along with content. What tool you are using for animations?
  • @
    @orisueXtriumvir7 months ago What techniques can we use for an endpoint that handles file download requests where the data content can change for each request?
  • @
    @TariqSajid9 months ago Hidden thing about pagnation is be very careful of the total count query that might be taking too much time if you do on every pagination request and you have large dataset. 6
  • @
    @LNSFLIVE5 months ago What you using to make youranimations are great.
  • @
    @danregep46469 months ago What about avoiding memcopy and serialization by using an optimizedformat? Trift pbufers etc)
  • @
    @SalvadorFaria9 months ago Other optimization techniques: partial responses and field masks - request specific data fields, reduces processing load, and improves efficiency in api interactions. 13
  • @
    @efnobrega2 months ago Hithere! Please, let me know what tool do you use to create the videos graphics / animations. Ths!
  • @
    @CarlosGonzalez-rg6ht9 months ago I think dataline to the querying to be performed could be a usefull method to improve performance.
  • @
    @hemantpanchal80879 months ago Can someone please help me in below scenario.
    eg. My program has more than 1. 5 lakh employees records in database and this data won' t get so is it good idea to publish json of individual employee on azure blob/s3 and than from api we can read from here and display in front-end.
    i would like to know whether it will impact read performance for 1. 5 lakh json on azure blob
    .
    ...Expand
  • @
    @Ashirgaziyev5 months ago Hey guys, i know this might be a silly question; however, what would be the great learning material for a newbie backend developer?
    i am considering . ...Expand
  • @
    @iSerjioL4 months ago I would point out the 3rd technique " avoid n+1 problem" into a separate section - database query optimisation, including proper indexes, query . ...Expand
  • @
    @JIANGNIANHANG8 months ago I have 2
    5. Json serialization
    which json library should been used?
    does json libraries has really large difference in performance?
    7. Asynchronous logging
    dose logging method can improve api performance?
    .
    ...Expand 1
  • @
    @infomaniac509 months ago When doing compression on api responses, make sure you' re not exposing yourself to a crime attack. Cve-2012-4929. 65
  • @
    @Mo-bs7ct9 months ago You may need indexing to speed up queries. 3
  • @
    @BhaveshAgarwal8 months ago Bytebytego - please share the tools and softwares you use to create these wonderful videos. It will be extremely helpful to learn them and use it for work and share knowledge in general. Thanks in advance!
  • @
    @Achrafsouk9 months ago Cdns like cloudfront also can help with connection pooling. 2
  • @
    @vighnesh1539 months ago You could also consider replacing json with protobufs. They are super optimized for data transfer between systems. 24
  • @
    @TomDoesTech9 months ago Anyone know how he makes these animations? 1
  • @
    @diegofelipe916 months ago In mongodb it' s not good to use skip and limit approach to pagination, specially when you' re paginating over a huge number of documents, it' s more suitable to have a find and limit approach instead.
  • @
    @andrewwhitehouse18789 months ago This is gold. Your whole channel is gold and the production values are amazing. 1
  • @
    @user-xe9on9yr7c8 months ago What about binary optimization and grpc?
  • @
    @user-ns2fz1tl9s9 months ago Pagination could be very difficult and confusion on some databases. For example postgres have to read all previous pages, to read 101 page. So on really big datasets just limit\offset leads only to problems.
  • @
    @sarakhushi236 months ago 1- caching, store result comoutation so that can be used later in redis.
    2- connection pool-
    3- avoid n+1 query problem
    4- pagination 5- lightweight json
    6- compression.
    7- asynchronous logging
    .
    ...Expand
  • @
    @aravind.a9 months ago Hi team, can you please explain serialization in detail?
    does it mean - instead of sending the json / xml it is better to send as string?
  • @
    @andrasczinege9 months ago Thanks for the awesome video. I use azure functions that often read and write from/to a database, and i am just wondering what if i create a connection . ...Expand
  • @
    @rajatmathur9 months ago Can you share some fast json serialization libraries for java?
  • @
    @_sk_videos8 months ago Any suggestions on how to improve an api that returns a large amount of data, 10-200mb?
  • @
    @olhoTron3 months ago Its worth stressing: always measure, before, during and after an optimization, and check if the performance really improved and the results remained the also think of the system as a whole, sometimes what shows up at the top on the profiler is the symptom and not the cause of the problem. ...Expand 1
  • @
    @myrondai8 months ago Use uncrowded resources for crowded resources: computing for(compression) memory for computing (cache)
    reuse objects;
    avoid unnecessary calculations;
    question: why do people want to do pagination? " user interface has limited place to display data, so we only fetch what we needed. " is one explanation. Anything else?
    .
    ...Expand
  • @
    @ColossalMcBuzz7 months ago Tip #8: ensure you' re using the right data structures and/or algorithms for the endpoints.
  • @
    @114dev89 months ago Hey guys, i was wondering if anyone can propose some good database architectures that can help to improve.
  • @
    @TheOnlyEpsilonAlpha4 months ago Other techniques:
    1. Key pool for oauth operations:
    - when you want to reduce the waste of individual overhead, that comes from the whole - by letting a dedicated component manage those pool for you, keep an eye that are always enough valid entries available. Based on the need the component can request a bigger amount of keys
    - the life-cycle of that keys and be easily managed by storing them in a redis database with expire set, so it will get deleted when it' s expired also in the according api
    .
    ...Expand
  • @
    @Rscnry999 months ago @bytebytego what app do you use for your animations?
  • @
    @phoenicianathletix28669 months ago Can payload compression be used while using web sockets? 1
  • @
    @robl399 months ago Fun fact: doing pagination killed performance at the database level for my product. We were using offset fetch in sql server and we quickly going out that . ...Expand 1
  • @
    @zakraw9 months ago Surprised you didn' t mention eager vs lazy loading, database indexing, breaking excessive sql queries with multi-level subqueries into smaller ones. 3
  • @
    @santosh_bhat3 months ago Someone please tell me how to make such animations?
  • @
    @jwbonnett9 months ago Json is the old defacto, protobuff is the new one.
  • @
    @LCTesla7 months ago kinda kills the idea for me. that's when logs are most important
  • @
    @globalcitizen123-hl3dv9 months ago Wait i had no idea you wrote the system design books hahaha damn, i' m stupid. I' m a technical program manager that works with platform/infra teams . ...Expand
  • @
    @nixonrod9 months ago Master slave | partitioning | sharding techniques for improving database performance.
  • @
    @jon91033 months ago Paging is often poorly implemented in practice because the ordering of items does not stay consistent between page loads leading to some items being missed and others duplicated and a jarring ux.
  • @
    @rollinOnCode9 months ago The n+1 query can be huge and tends to happen when you got scope creep.
  • @
    @Nephtys19 months ago This is missing the number 0 of optimization: deleting your api / get rid of it entirely. Nothing is faster or cheaper than no code.
    contrary to last year i optimized a specificaway by moving its(nearly nothing really useful) to the client. Decreased latency of the whole feature by 80% and decreased total loc.
    always try to localize operations or data if your goal is speed and efficiency
    .
    ...Expand
  • @
    @amancca9 months ago Image compression is one of my favorite one. My api can reduce image from 5mb to 10kb. 3
  • @
    @VadimFilin9 months ago Do not use offset! Use cursor pattern instead. 4