Details: PROC ENTROPY :: SAS/ETS(R) 12.1 User's Guide

Details: ENTROPY Procedure

Generalized Maximum Entropy
Generalized Cross Entropy
Moment Generalized Maximum Entropy
Maximum Entropy-Based Seemingly Unrelated Regression
Generalized Maximum Entropy for Multinomial Discrete Choice Models
Censored or Truncated Dependent Variables
Information Measures
Parameter Covariance For GCE
Parameter Covariance For GCE-M
Statistical Tests
Missing Values
Input Data Sets
Output Data Sets
ODS Table Names
ODS Graphics

Shannon’s measure of entropy for a distribution is given by

$\displaystyle \mr {maximize}$	$\displaystyle$	$\displaystyle - \sum _{i=1}^{n} \, p_{i} \, \ln (p_{i})$
$\displaystyle \mr {subject\, to}$	$\displaystyle$	$\displaystyle \sum _{i=1}^{n} \, p_{i}\, = \, 1$

where $p_{i}$ is the probability associated with the ith support point. Properties that characterize the entropy measure are set forth by Kapur and Kesavan (1992).

The objective is to maximize the entropy of the distribution with respect to the probabilities $p_{i}$ and subject to constraints that reflect any other known information about the distribution (Jaynes, 1957). This measure, in the absence of additional information, reaches a maximum when the probabilities are uniform. A distribution other than the uniform distribution arises from information already known.

The ENTROPY Procedure (Experimental)

Details: ENTROPY Procedure