So I'm reading this paper which uses a max ent classifier for sentence extraction. The parametric form for a conditional max ent model is :

$$ P(c|s) = \frac{1}{Z(s)} \exp \sum_i \lambda_if_i(c,s)$$

where $f_i(c,s)$ is a feature which has a weight $\lambda_i$ associated with it.

Now, the paper states that conjugate descent was used to find the optimal set of weights (page 3) - and this is what I'm unable to comprehend. How to use/apply conjugate gradient descent to calculate the optimal set of weights?


Conjugate gradient descent is a variation on gradient descent. Gradient descent is a method of finding the minimum of a function by taking steps that reduce the error between the data and the model parameters. Conjugate gradient descent extends gradient descent by searching a plane, instead of a line. The plane is defined as a linear combination of the gradient vector and the previous descent step vector. Conjugate gradient descent is very good at finding the solutions to a set of sparse linear equations.

Gradient descent and variations are general methods for finding the best parameters. The best practice is to define your specific model and then call on a separate gradient descent package to search for optimal values (i.e., hyperparameters) for the model.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.