what does regularization mean in xgboost (tree)












3












$begingroup$


In xgboost (xgbtree), gamma is the tunning parameter to control the regularization. I understand what regularization means in xgblinear and logistic regression, but in the context of tree-based methods, I'm not sure how regularization works.



Can someone explain how regularization works in xgbtree?






r regularization xgboost







share|cite|improve this question











$endgroup$

















    3












    $begingroup$


    In xgboost (xgbtree), gamma is the tunning parameter to control the regularization. I understand what regularization means in xgblinear and logistic regression, but in the context of tree-based methods, I'm not sure how regularization works.



    Can someone explain how regularization works in xgbtree?






    r regularization xgboost







    share|cite|improve this question











    $endgroup$















      3












      3








      3





      $begingroup$


      In xgboost (xgbtree), gamma is the tunning parameter to control the regularization. I understand what regularization means in xgblinear and logistic regression, but in the context of tree-based methods, I'm not sure how regularization works.



      Can someone explain how regularization works in xgbtree?






      r regularization xgboost







      share|cite|improve this question











      $endgroup$




      In xgboost (xgbtree), gamma is the tunning parameter to control the regularization. I understand what regularization means in xgblinear and logistic regression, but in the context of tree-based methods, I'm not sure how regularization works.



      Can someone explain how regularization works in xgbtree?






      r regularization xgboost




      r regularization xgboost






      share|cite|improve this question















      share|cite|improve this question













      share|cite|improve this question




      share|cite|improve this question








      edited 6 hours ago









      kiamlaluno

      1034




      1034










      asked 10 hours ago









      zeslazesla

      20218




      20218






















          1 Answer
          1






          active

          oldest

          votes


















          2












          $begingroup$

          In tree-based methods regularization is usually understood as defining a minimum gain so which another split happens:




          Minimum loss reduction required to make a further partition on a leaf
          node of the tree. The larger gamma is, the more conservative the
          algorithm will be.




          Source: https://xgboost.readthedocs.io/en/latest/parameter.html



          This minimum gain can usually be set for anything between $(0,infty)$.



          Here's a somewhat good article on how to tune regularization on XGBoost.






          share|cite|improve this answer











          $endgroup$













            Your Answer





            StackExchange.ifUsing("editor", function () {
            return StackExchange.using("mathjaxEditing", function () {
            StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
            StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
            });
            });
            }, "mathjax-editing");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "65"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: false,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: null,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f392971%2fwhat-does-regularization-mean-in-xgboost-tree%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            2












            $begingroup$

            In tree-based methods regularization is usually understood as defining a minimum gain so which another split happens:




            Minimum loss reduction required to make a further partition on a leaf
            node of the tree. The larger gamma is, the more conservative the
            algorithm will be.




            Source: https://xgboost.readthedocs.io/en/latest/parameter.html



            This minimum gain can usually be set for anything between $(0,infty)$.



            Here's a somewhat good article on how to tune regularization on XGBoost.






            share|cite|improve this answer











            $endgroup$


















              2












              $begingroup$

              In tree-based methods regularization is usually understood as defining a minimum gain so which another split happens:




              Minimum loss reduction required to make a further partition on a leaf
              node of the tree. The larger gamma is, the more conservative the
              algorithm will be.




              Source: https://xgboost.readthedocs.io/en/latest/parameter.html



              This minimum gain can usually be set for anything between $(0,infty)$.



              Here's a somewhat good article on how to tune regularization on XGBoost.






              share|cite|improve this answer











              $endgroup$
















                2












                2








                2





                $begingroup$

                In tree-based methods regularization is usually understood as defining a minimum gain so which another split happens:




                Minimum loss reduction required to make a further partition on a leaf
                node of the tree. The larger gamma is, the more conservative the
                algorithm will be.




                Source: https://xgboost.readthedocs.io/en/latest/parameter.html



                This minimum gain can usually be set for anything between $(0,infty)$.



                Here's a somewhat good article on how to tune regularization on XGBoost.






                share|cite|improve this answer











                $endgroup$



                In tree-based methods regularization is usually understood as defining a minimum gain so which another split happens:




                Minimum loss reduction required to make a further partition on a leaf
                node of the tree. The larger gamma is, the more conservative the
                algorithm will be.




                Source: https://xgboost.readthedocs.io/en/latest/parameter.html



                This minimum gain can usually be set for anything between $(0,infty)$.



                Here's a somewhat good article on how to tune regularization on XGBoost.







                share|cite|improve this answer














                share|cite|improve this answer



                share|cite|improve this answer








                edited 9 hours ago

























                answered 9 hours ago









                Lucas FariasLucas Farias

                642421




                642421






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Cross Validated!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid …



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    Use MathJax to format equations. MathJax reference.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f392971%2fwhat-does-regularization-mean-in-xgboost-tree%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    -

                    Popular posts from this blog

                    Усть-Каменогорск

                    Image
                    У этого термина существуют и другие значения, см. Усть-Каменогорск. Город Усть-Каменогорск каз. Өскемен Флаг Герб 49°57′ с. ш. 82°37′ в. д. H G Я O L Страна Казахстан   Казахстан Статус областной центр Область Восточно-Казахстанская Аким Жаксылык Омар [1] История и география Основан 1720 Город с 1868 Площадь 303,15 [2] км² Высота центра 283 ± 1 м Тип климата резко-континентальный Часовой пояс UTC+6 Население Население 329 090 [3]  человек ( 2018 ) Национальности русские 53,56 % казахи 42,99 % немцы 0,82 % украинцы 0,57 % татары 0,79 % белорусы 0,14 % азербайджанцы 0,23 % корейцы 0,20 % прочие 1,08 % -1 шт [4] Этнохороним усть-каменогорцы, усть-каменогорец Цифровые идентификаторы Телефонный код +7 (7232) Почтовый индекс F0***** [5] Автомобильный код 16 (ранее F) oskemen.kz Усть-Каменогорск Усть-Каменогорск ...
                    Read more

                    Халкинская богословская школа

                    Image
                    Халкинская богословская школа тур. Rum Ortodoks Ruhban Okulu греч. Ιερά Θεολογική Σχολή της Χάλκης Главное здание Халкинской школы Международное название Theological School of Halki Год основания 1844 Конфессия православие Церковь Константинопольская Ректор Елпидифор (Ламбриниадис) Расположение о. Хейбелиада, Турция Юридический адрес Ümit tepesi No 5, 34973 Heybeliada, Istanbul, Turkey Сайт www.greece.org  [[commons:Category:Halki seminary|Халкинская богословская школа тур. Rum Ortodoks Ruhban Okulu греч. Ιερά Θεολογική Σχολή της Χάλκης на Викискладе]] Ха́лкинская богосло́вская шко́ла (или Духовная семинария на Халки , тур. Rum Ortodoks Ruhban Okulu , греч. Ιερά Θεολογική Σχολή της Χάλκης ) — недействующее среднее специальное учебное заведение Константинопольской православной церкви, расположенное на территории Свято-Троицкого мужского монастыря на вершине холма Надежды (тур. Ümit Tepesi ) острова Хейбелиада, в Турции....
                    Read more

                    Высокополье (Харьковская область)

                    Image
                    У этого термина существуют и другие значения, см. Высокополье. Село Высокополье укр. Високопілля Герб 49°55′07″ с. ш. 35°25′13″ в. д. H G Я O L Страна Украина   Украина Область Харьковская Район Валковский Сельский совет Высокопольский История и география Основан 1670 Прежние названия Хмелевое Площадь 55,56 км² Часовой пояс UTC+2, летом UTC+3 Население Население 1957 человек ( 2001 ) Плотность 35,220 чел./км² Цифровые идентификаторы Телефонный код +380 5753 Почтовый индекс 63020 Автомобильный код AX, КХ / 21 КОАТУУ 6321281501 Высокополье Высокополье Высокополье  Аудио, фото и видео на Викискладе Высокопо́лье (укр. Високопілля ) — село, Высокопольский сельский совет, Валковский район, Харьковская область, Украина. Код КОАТУУ — 6321281501. Население по переписи 2001 г. составляет 1957 (866/1091 м/ж) человек. Является административным центр...
                    Read more