Rich Pang personal blog
An alternative construction of Shannon entropy
The underpinning of information theory is Shannon’s formula for the entropy H of a discrete probability distribution P. Derived in 1948 in the paper “A mathematical theory of communication”, this is given by \[ H[P] = -\sum_{i=1}^N p_i \log p_i \] where \( p_i = p(x_i) \) is the probability of sampling \( x_i \) from…
- Points
- 126
- HN comments
- 11
- Submissions
- 1
- Cited by
- 0 commenters
On Hacker News
| Submitted | Title on HN | Points | Comments |
|---|---|---|---|
| Nov 13, 2024 | An alternative construction of Shannon entropy by rkp8000 | 126 | 11 |
