This paper is all about the problem of Rule Ensemble Learning (REL) using Hierarchical Kernel Learning, where the goal is simultaneous discovery of a small set of simple rules and their optimal weights that lead to good generalization. Rules are assumed to be conjunctions of basic propositions concerning the values taken by the input features. From the perspectives of interpretability as well as generalization, it is highly desirable to construct rule ensembles with low training error, having rules that are i) simple, i.e., involve few conjunctions and ii) few in number. We propose to explore the (exponentially) large feature space of all possible conjunctions optimally and efficiently by employing the recently introduced Hierarchical Kernel Learning (HKL) framework.
The regularizes employed in the Hierarchical Kernels formulation can be interpreted as a potential for discouraging selection of rules involving large number of conjunctions { justifying its suitability for constructing rule ensembles. Simulation results show that, in case of many benchmark data sets, the proposed approach improves over state-of-the-art REL algorithms in terms of generalization and indeed learns simple rules. Unfortunately, Hierarchical Kernels selects a conjunction only if all its subsets are selected.We propose a novel convex formulation which alleviates this problem and generalizes the Hierarchical Kernels framework.
The main technical contribution of this paper is an efficient mirror descent based active set algorithm for solving the new formulation. Empirical evaluations on REL problems illustrate the utility of generalized using Hierarchical Kernels.
You can also Subscribe to PROJECTSWORLDS by Email for more such projects and seminar.
You can also Subscribe to PROJECTSWORLDS by Email for more such projects and seminar.
For more updates on Projects via E-mail or Sms Subscribe to www.projectsworlds.blogspot.com
