Cross Entropy (CE) method is a general Monte Carlo method originally proposed to estimate rare-event probabilities but then naturally extended to solve optimization problems. It is relevant to several my previous posts. For example, both Bayesian Optimization [5] and CE method can be used to solve black-box optimization problems, although Bayesian Optimization mostly works on continuous input …
Category Archives: Algorithm
My understanding in Bayesian Optimization
In this post, I am going to share my understanding in Bayesian Optimization. Bayesian Optimization is closely related Gaussian Process, a probabilistic model explained in details in a textbook called “Gaussian Processes for Machine Learning” [1]. In the following, “p. ” will refer to a specific page in [1] where the content comes from. First, let’s dive into the …
Continue reading “My understanding in Bayesian Optimization”
Combinatorial Optimization using Pointer Network (Code Walkthrough)
In this post, I am going to walk through an online piece of code [7] which implements the idea of [1]: using pointer network [2] to solve travelling salesman problem. Pointer networks, in my understanding, are neural network architectures for the problems where output sequences come from the permutation of input sequences. Some background posts …
Continue reading “Combinatorial Optimization using Pointer Network (Code Walkthrough)”
Revisit Support Vector Machine
This post reviews the key knowledgement about support vector machine (SVM). The post is pretty much based on a series of the lectures [1]. Suppose we have data , where each point is a -dimensional data point associated with label . The very initial SVM is to come up with a plane such that data …
Revisit Traveling Salesman Problem
Travelling salesman problem (TSP) goes as follows [1]: Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city and returns to the origin city? This problem statement is actually a TSP-OPT problem., where”OPT” stands for optimization. This is not a decision problem …
Reinforcement Learning in Web Products
Reinforcement learning (RL) is an area of machine learning concerned with optimizing a notion of cumulative rewards. Although it has been applied in video game AI, robotics and control optimization for years, we have seen less of its presence in web products. In this post, I am going to introduce some works that apply RL in …
LambdaMART
LambdaMART [7] is one of Learn to Rank algorithms. It emphasizes on fitting on the correct order of a list, which contains all documents returned by a query and marked as different relevance. It is a derivation/combination of RankNet, LambdaRank and MART (Multiple Addictive Regression Tree). For me, the most helpful order to approach LambdaMART is …
Questions on Guided Policy Search
I’ve been reading Prof. Sergey Levine‘s paper on Guided Policy Search (GPS) [2]. However, I do not understand about it but want to have a record of my questions so maybe in the future I could look back and solve. Based on my understanding, traditional policy search (e.g., REINFORCE) maximizes the likelihood ratio of rewards. This …
Relationships between DP, RL, Prioritized Sweeping, Prioritized Experience Replay, etc
In the last weekend, I’ve struggled with many concepts in Reinforcement Learning (RL) and Dynamic Programming (DP). In this post, I am collecting some of my thoughts about DP, RL, Prioritized Sweeping and Prioritized Experience Replay. Please also refer to a previous post written when I first learned RL. Let’s first introduce a Markov Decision …
Time Series Walk Through
In this post, I am going to give a practicum walk through on time series analysis. All related code is available in a python notebook. The data we use is International airline passengers: monthly totals in thousands. which can be downloaded here as csv file (on the left panel, select Export->csv(,)). It is a univariate time …