Orthogonal functions: Difference between revisions

Latest revision as of 19:15, 28 September 2014

Signals and Systems

Introduction

In this article we will examine another viewpoint for functions than that traditionally taken. Normally we think of a function $f(t)\,\!$ as a complicated entity $f()\,\!$ in a simple environment (one dimension, or along the t axis). Now we want to think of a function as a vector or point (a simple thing) in a very complicated environment (possibly an infinite dimensional space).

Vectors

Recall that vectors consist of an ordered set of numbers. Often the numbers are Real numbers, but we shall allow them to be Complex for our purposes. The numbers represent the amount of the vector in the direction denoted by the position of the number in the list. Each position in the list is associated with a direction. For example, the vector ${\vec {\mathbf {v}}}=<1,4,3>$ means that the vector ${\vec {\mathbf {v}}}$ is one unit in the first direction (often the x direction), four units in the second direction (often the y direction), and three units in the last direction (often the z direction). We say the component of ${\vec {\mathbf {v}}}$ in the second direction is 4. This is often written as $v_{y}=4\,\!$ .

Vector notation

We don't have to use x, y, and z as the direction names; we can use numbers, like 1, 2, and 3 instead. The advantage of this is that it leads to more compact notation, and extends to more than three dimensions much better. For example we could say $v_{2}=4\,\!$ instead of $v_{y}=4\,\!$ . Instead of writing ${\vec {\mathbf {v}}}=<1,4,3>$ we can write ${\vec {\mathbf {v}}}=\sum _{k=1}^{3}v_{k}{\hat {\mathbf {a}}}_{k}$ where the ${\hat {\mathbf {a}}}_{k}$ denotes a basis vector in the kth direction, $v_{1}=1,\,\!$ $v_{2}=4,\,\!$ and $v_{3}=3\,\!$ . The idea of basis vectors was implicit in the notation ${\vec {\mathbf {v}}}=<1,4,3>$ .

Inner products for vectors

When vectors are real, inner products (sometimes called dot products) give the component of one vector in another vector's direction, scaled by the magnitude (length) of the second vector. Inner products are useful to find components of vectors. We commonly use a dot as the symbol for inner product. For example, the inner product of ${\vec {\mathbf {v}}}$ and ${\vec {\mathbf {a}}}_{n}$ is written:

${\vec {\mathbf {v}}}\cdot {\vec {\mathbf {a}}}_{n}$

Orthogonality for vectors

It is quite handy to pick the directions used so that they are perpendicular (or orthogonal). With this arrangement the basis vectors have no components in each other's directions, which means that

${\vec {\mathbf {a}}}_{k}\cdot {\vec {\mathbf {a}}}_{n}=w_{k}\delta _{k,n}$

where the $w_{k}\,\!$ is the square of the length of ${\vec {\mathbf {a}}}_{k}$ and the symbol $\delta _{k,n}\,\!$ , known as the Kronecker delta, is one when k = n and zero otherwise.

Normalization

When the $w_{k}=1\,\!$ we have an orthonormal basis set, meaning that it is both orthogonal and that the basis vectors are normalized to unity (or have length one). Orthonormal vector systems are very popular. In fact they are the most common vector systems you will find. The reason they are so handy is each direction is uncoupled from the others.

For example, to find $v_{n}\,\!$ , we take the inner product of the vector ${\vec {\mathbf {v}}}$ with a unit vector in the nth direction, ${\vec {\mathbf {a}}}_{n}$ . We write this operation like this:

${\vec {\mathbf {v}}}\cdot {\vec {\mathbf {a}}}_{n}=\sum _{k=1}^{3}v_{k}{\vec {\mathbf {a}}}_{k}\cdot {\vec {\mathbf {a}}}_{n}=\sum _{k=1}^{3}v_{k}\delta _{k,n}=v_{n}$

Suppose we have two vectors from an orthonormal system, ${\vec {\mathbf {u}}}$ and ${\vec {\mathbf {v}}}$ . Taking the inner product of these vectors, we get

${\vec {\mathbf {u}}}\cdot {\vec {\mathbf {v}}}=\sum _{k=1}^{3}u_{k}{\vec {\mathbf {a}}}_{k}\cdot \sum _{m=1}^{3}v_{m}{\vec {\mathbf {a}}}_{m}=\sum _{k=1}^{3}u_{k}\sum _{m=1}^{3}v_{m}{\vec {\mathbf {a}}}_{k}\cdot {\vec {\mathbf {a}}}_{m}=\sum _{k=1}^{3}u_{k}\sum _{m=1}^{3}v_{m}\delta _{k,m}=\sum _{k=1}^{3}v_{k}u_{k}$

This shows that when we have an orthonormal vector space, inner products boil down to summing the products of like components. Also note that if we take the inner product of ${\vec {\mathbf {v}}}$ with itself, we get

${\vec {\mathbf {v}}}\cdot {\vec {\mathbf {v}}}=\sum _{k=1}^{3}v_{k}{\vec {\mathbf {a}}}_{k}\cdot \sum _{m=1}^{3}v_{m}{\vec {\mathbf {a}}}_{m}=\sum _{k=1}^{3}v_{k}\sum _{m=1}^{3}v_{m}{\vec {\mathbf {a}}}_{k}\cdot {\vec {\mathbf {a}}}_{m}=\sum _{k=1}^{3}v_{k}\sum _{m=1}^{3}v_{m}\delta _{k,m}=\sum _{k=1}^{3}v_{k}^{2}$

which is the magnitude of the vector ${\vec {\mathbf {v}}}$ squared ( $|{\vec {\mathbf {v}}}|^{2}$ ) from the Pythagorean Theorem.

Changing vector basis sets

Sometimes in our studies we find it useful to change basis sets. For example, when solving a physics problem with cylindrical symmetry, it is often easier to use cylindrical coordinates, and the basis vectors that go with that system, rather than the more usual Cartesian coordinates and basis vectors.

So, how do I change the basis set?

If the new basis set is orthonormal, it is really pretty simple. You need to project the vector you want changed onto each of the new basis vectors. This means that the new components are just the inner product of the vector and the appropriate basis function. If the new basis set is not orthonormal, and if there are n dimensions in each basis set, you will have n linear coupled equations in n unknowns to solve.

Functions and vectors, an analogy

We may think of the number of the direction, $k\,\!$ , as the independent variable of a vector and the component in that direction, $v_{k}\,\!$ as the dependent variable of the vector ${\vec {\mathbf {v}}}$ in a similar way to the way we think of $t\,\!$ as the independent variable of a function $f()\,\!$ , where $f(t)\,\!$ is the dependent variable of $f\,\!$ . Probably the biggest difference here is that t often takes on real values from $-\infty$ to $\infty$ , and $k\in {1,2,3}$ . Using this analogy, we may think of a function as a vector having an uncountably infinite number of dimensions.

Can we write functions in an analogous way compared to the way we write vectors?

Remember we wrote ${\vec {\mathbf {v}}}=\sum _{k=1}^{3}v_{k}{\hat {\mathbf {a}}}_{k}$ . Can we write something similar for a function, $f(t)\,\!$ defined for a $t\,\!$ element of the reals? Well maybe.... If the sum over the dummy index $k\,\!$ becomes an integral over the dummy variable, $x\,\!$ , and the unit vectors ${\vec {\mathbf {a}}}_{k}$ are replaced with something like $\delta (x-t)\,\!$ , the Dirac delta function. The result would look something like this:

$f(t)=\int _{-\infty }^{\infty }f(x)\delta (x-t)dx$ .

This works! The Dirac delta functions, playing the roll of the basis vectors, are called basis functions. The function f(x) plays the roll of the vector coefficients $v_{k}$ . This gives us another way to think of the function f().

Inner products for functions

Above we found that a vector inner product between ${\vec {\mathbf {u}}}$ and ${\vec {\mathbf {v}}}$ could be written as ${\vec {\mathbf {u}}}\cdot {\vec {\mathbf {v}}}=\sum _{k=1}^{3}u_{k}v_{k}$ . If we follow our above analogy, we should be able to replace the sum over k with an integral over x. There is one little notational problem, and that is we don't want to confuse the functional inner product with a simple muliply, so we need some new notation to denote this new inner product. In quantum mechanics, physicists use the bra-ket notation. Let's borrow that.

$<u|v>=\int _{-\infty }^{\infty }u^{*}(x)v(x)dx$

Note the complex conjugate on the function u(x). That is in case u(x) is a complex valued function. For the analogous case with vectors see Complex vector inner products.

Orthogonality for functions

Two functions, $u(t)\!$ and $v(t)\!$ are said to be orthogonal on the interval $(a,b)\!$ with respect to the weighting function $w(t)\!$ if and only if $\int _{a}^{b}w(x)u^{*}(x)v(x)dx=0\!$ . The weighting function is often unity, but it is included so that different values of $t\!$ can be weighted appropriately in analogy to the way the $w_{k}\!$ weight was used when the vector basis set was orthogonal, but not orthonormal (that is, different basis vectors had different numerical lengths), as we discussed here. Unless otherwise noted we will use $w(t)=1\!$ , so that the defining relation for orthogonality of functions $u\!$ and $v\!$ becomes $\int _{a}^{b}u^{*}(x)v(x)dx=0$ .

Changing basis sets with functions

Examples

Other resources on orthogonality

Another professor's view of this topic...

Wikipedia inner product

Wikipedia Orthogonality

Principle author of this page: Rob Frohne

@@ Line 1: / Line 1: @@
+*[[Signals and systems|Signals and Systems]]
 ==Introduction==
-In this article we will examine another viewpoint for functions than that traditionally taken.  Normally we think of a function <math> f(t) </math> as a complicated entity - <math> f() </math> in a simple environment (one dimension, or along the t axis).  Now we want to think of a function as a vector or point (a simple thing) in a very complicated environment (possibly an infinite dimensional space).
+In this article we will examine another viewpoint for functions than that traditionally taken.  Normally we think of a function <math> f(t) \,\!</math> as a complicated entity <math> f() \,\!</math> in a simple environment (one dimension, or along the t axis).  Now we want to think of a function as a vector or point (a simple thing) in a very complicated environment (possibly an infinite dimensional space).
 ==Vectors==
-Recall that vectors consist of an ordered set of numbers.  Often the numbers are Real numbers, but we shall allow them to be from the Complex numbers for our purposes.  The numbers represent the amount of the vector in the direction denoted by the position of the number in the list.  Each position in the list is associated with a direction.  For example, the vector
+Recall that vectors consist of an ordered set of numbers.  Often the numbers are Real numbers, but we shall allow them to be Complex for our purposes.  The numbers represent the amount of the vector in the direction denoted by the position of the number in the list.  Each position in the list is associated with a direction.  For example, the vector
-<math> \vec \bold v = <1, 4, 3> </math> means that the vector <math> \vec \bold v </math> is one unit in the first direction (often the x direction), four units in the second direction (often the y direction), and three units in the last direction (often the z direction).  We say the component of <math> \vec \bold v </math> in the second direction is 4.  This is often written as <math> v_y = 4 </math>.
+<math> \vec \bold v = <1, 4, 3> </math> means that the vector <math> \vec \bold v </math> is one unit in the first direction (often the x direction), four units in the second direction (often the y direction), and three units in the last direction (often the z direction).  We say the component of <math> \vec \bold v </math> in the second direction is 4.  This is often written as <math> v_y = 4 \,\!</math>.
 ====Vector notation====
-We don't have to use x, y, and z as the direction names; we can use numbers, like 1, 2, and 3 instead.  The advantage of this is that it leads to more compact notation, and extends to more than three dimensions much better.  For example we could say <math> v_2 = 4 </math> instead of <math> v_y = 4 </math>.  Instead of writing <math> \vec \bold v = <1, 4, 3> </math> we can write <math> \vec \bold v = \sum_{k=1}^3 v_k \hat \bold a_k </math> where the <math>\hat \bold a_k </math> denotes a basis vector in the kth direction, <math>v_1 = 1,</math> <math> v_2 = 4, </math> and <math> v_3 = 3</math>.  The idea of basis vectors was implicit in the notation <math> \vec \bold v = <1, 4, 3> </math>.
+We don't have to use x, y, and z as the direction names; we can use numbers, like 1, 2, and 3 instead.  The advantage of this is that it leads to more compact notation, and extends to more than three dimensions much better.  For example we could say <math> v_2 = 4 \,\!</math> instead of <math> v_y = 4 \,\!</math>.  Instead of writing <math> \vec \bold v = <1, 4, 3> </math> we can write <math> \vec \bold v = \sum_{k=1}^3 v_k \hat \bold a_k </math> where the <math>\hat \bold a_k </math> denotes a basis vector in the kth direction, <math>v_1 = 1, \,\!</math> <math> v_2 = 4, \,\!</math> and <math> v_3 = 3 \,\!</math>.  The idea of basis vectors was implicit in the notation <math> \vec \bold v = <1, 4, 3> </math>.
 ===Inner products for vectors===
-When vectors are real, inner products, sometimes called dot products give the component of one vector in another vector's direction, scaled by the magnitude (length) of the second vector.  Inner products are useful to find components of vectors.  We commonly use a dot as the symbol for inner product.  For example, the inner product of <math> \vec \bold v </math> and <math> \vec \bold a_n </math> is written:
+When vectors are real, inner products (sometimes called dot products) give the component of one vector in another vector's direction, scaled by the magnitude (length) of the second vector.  Inner products are useful to find components of vectors.  We commonly use a dot as the symbol for inner product.  For example, the inner product of <math> \vec \bold v </math> and <math> \vec \bold a_n </math> is written:
-<math> \vec \bold v \bullet \vec \bold a_n </math>
+<math> \vec \bold v \cdot \vec \bold a_n </math>
 ====Orthogonality for vectors====
 It is quite handy to pick the directions used so that they are perpendicular (or orthogonal).  With this arrangement the basis vectors have no components in each other's directions, which means that
-<math>\vec \bold a_k \bullet \vec \bold a_n = w_k \delta_{k,n} </math>
+<math>\vec \bold a_k \cdot \vec \bold a_n = w_k \delta_{k,n} </math>
-where the <math> w_k </math> is the square of the length of <math> \vec \bold a_k </math> and the symbol <math> \delta_{k,n} </math>, known as the [http://en.wikipedia.org/wiki/Kronecker_delta Kronecker delta], is one when k = n and zero otherwise.
+where the <math> w_k \,\!</math> is the square of the length of <math> \vec \bold a_k </math> and the symbol <math> \delta_{k,n} \,\!</math>, known as the [http://en.wikipedia.org/wiki/Kronecker_delta Kronecker delta], is one when k = n and zero otherwise.
 =====Normalization=====
-When the <math> w_k = 1</math> we have an orthonormal basis set, meaning that it is both orthogonal and that the basis vectors are normalized to unity (or have length one).  Orthonormal vector systems are very popular.  In fact they are the most common vector systems you will find.  The reason they are so handy is each direction is uncoupled from the others.
+When the <math> w_k = 1 \,\!</math> we have an orthonormal basis set, meaning that it is both orthogonal and that the basis vectors are normalized to unity (or have length one).  Orthonormal vector systems are very popular.  In fact they are the most common vector systems you will find.  The reason they are so handy is each direction is uncoupled from the others.
-For example, to find <math> v_n </math>, we take the inner product of the vector <math> \vec \bold v </math> with a unit vector in the nth direction, <math> \vec \bold a_n </math>.  We write this operation like this:
+For example, to find <math> v_n \,\!</math>, we take the inner product of the vector <math> \vec \bold v </math> with a unit vector in the nth direction, <math> \vec \bold a_n </math>.  We write this operation like this:
-<math> \vec \bold v \bullet \vec \bold a_n = \sum_{k=1}^3 v_k \vec \bold a_k \bullet \vec \bold a_n = \sum_{k=1}^3 v_k \delta_{k,n} =  v_n </math>
+<math> \vec \bold v \cdot \vec \bold a_n = \sum_{k=1}^3 v_k \vec \bold a_k \cdot \vec \bold a_n = \sum_{k=1}^3 v_k \delta_{k,n} =  v_n </math>
 Suppose we have two vectors from an orthonormal system, <math> \vec \bold u </math> and <math> \vec \bold v </math>.  Taking the inner product of these vectors, we get
-<math> \vec \bold u \bullet \vec \bold v = \sum_{k=1}^3 u_k \vec \bold a_k \bullet \sum_{m=1}^3 v_m \vec \bold a_m  = \sum_{k=1}^3 u_k \sum_{m=1}^3  v_m \vec \bold a_k \bullet  \vec \bold a_m = \sum_{k=1}^3 u_k \sum_{m=1}^3  v_m \delta_{k,m} = \sum_{k=1}^3 v_k u_k </math>
+<math> \vec \bold u \cdot \vec \bold v = \sum_{k=1}^3 u_k \vec \bold a_k \cdot \sum_{m=1}^3 v_m \vec \bold a_m  = \sum_{k=1}^3 u_k \sum_{m=1}^3  v_m \vec \bold a_k \cdot \vec \bold a_m = \sum_{k=1}^3 u_k \sum_{m=1}^3  v_m \delta_{k,m} = \sum_{k=1}^3 v_k u_k </math>
 This shows that when we have an orthonormal vector space, inner products boil down to summing the products of like components.  Also note that if we take the inner product of <math> \vec \bold v </math> with itself, we get
-<math> \vec \bold v \bullet \vec \bold v = \sum_{k=1}^3 v_k \vec \bold a_k \bullet \sum_{m=1}^3 v_m \vec \bold a_m  = \sum_{k=1}^3 v_k \sum_{m=1}^3  v_m \vec \bold a_k \bullet  \vec \bold a_m = \sum_{k=1}^3 v_k \sum_{m=1}^3  v_m \delta_{k,m} = \sum_{k=1}^3 v_k^2</math>
+<math> \vec \bold v \cdot \vec \bold v = \sum_{k=1}^3 v_k \vec \bold a_k \cdot \sum_{m=1}^3 v_m \vec \bold a_m  = \sum_{k=1}^3 v_k \sum_{m=1}^3  v_m \vec \bold a_k \cdot \vec \bold a_m = \sum_{k=1}^3 v_k \sum_{m=1}^3  v_m \delta_{k,m} = \sum_{k=1}^3 v_k^2</math>
 which is the magnitude of the vector <math> \vec \bold v </math> squared (<math> | \vec \bold v |^2 </math>) from the Pythagorean Theorem.
@@ Line 50: / Line 51: @@
 ==Functions and vectors, an analogy==
-We may think of the number of the direction, <math> k </math>, as the independent variable of a vector and the component in that direction, <math> v_k </math> as the dependent variable of the vector <math> \vec \bold  v </math> in a similar way to the way we think of t as the independent variable of a function f(), where f(t) is the dependent variable of f.  Probably the biggest difference here is that t often takes on real values from <math> - \infty </math> to <math> \infty </math>, and <math> k \in {1, 2, 3} </math>.  Using this analogy, we may think of a function as a vector having an uncountably infinite number of dimensions.
+We may think of the number of the direction, <math> k \,\!</math>, as the independent variable of a vector and the component in that direction, <math> v_k \,\!</math> as the dependent variable of the vector <math> \vec \bold  v </math> in a similar way to the way we think of <math> t \,\! </math> as the independent variable of a function <math> f() \,\! </math>, where <math> f(t)\,\! </math> is the dependent variable of <math> f\,\! </math>.  Probably the biggest difference here is that t often takes on real values from <math> - \infty </math> to <math> \infty </math>, and <math> k \in {1, 2, 3} </math>.  Using this analogy, we may think of a function as a vector having an uncountably infinite number of dimensions.
-====Can we write functions in an analogous way to the way we write vectors?====
+====Can we write functions in an analogous way compared to the way we write vectors?====
-Remember we wrote <math> \vec \bold v = \sum_{k=1}^3 v_k \hat \bold a_k </math>.  Can we write something similar for a function, f(t) defined for t an element of the reals?  Well maybe....  If the sum over the dummy index k becomes an integral over the dummy variable, x, and the unit vectors <math> \vec \bold a_k </math> are replaced with something like <math> \delta(x-t) </math>, the [http://en.wikipedia.org/wiki/Delta_function Dirac delta function].  The result would look something like this:
+Remember we wrote <math> \vec \bold v = \sum_{k=1}^3 v_k \hat \bold a_k </math>.  Can we write something similar for a function, <math> f(t)\,\! </math> defined for a <math> t \,\! </math> element of the reals?  Well maybe....  If the sum over the dummy index <math> k \,\!</math>becomes an integral over the dummy variable, <math> x \,\! </math>, and the unit vectors <math> \vec \bold a_k </math> are replaced with something like <math> \delta(x-t) \,\!</math>, the [http://en.wikipedia.org/wiki/Delta_function Dirac delta function].  The result would look something like this:
 <math> f(t) = \int_{- \infty}^\infty f(x) \delta (x-t) dx </math>.
 This works!  The Dirac delta functions, playing the roll of the basis vectors, are called basis functions.  The function f(x) plays the roll of the vector coefficients <math>v_k</math>.  This gives us another way to think of the function f().
 ===Inner products for functions===
-[[Orthogonal functions#Inner products for vectors|Above]] we found that a vector inner product between <math>\vec \bold u </math> and <math>\vec \bold v </math> could be written as <math> \vec \bold u \bullet \vec \bold v = \sum_{k=1}^3 u_k v_k </math>.  If we follow our above analogy, we should be able to replace the sum over k with an integral over x.  There is one little notational problem, and that is we don't want to confuse the functional inner product with a simple muliply, so we need some new notation to denote this new inner product.  In [http://en.wikipedia.org/wiki/Quantum_mechanics quantum mechanics], physicists use the [http://en.wikipedia.org/wiki/Bra-ket_notation bra-ket] notation.  Let's borrow that.
+[[Orthogonal functions#Inner products for vectors|Above]] we found that a vector inner product between <math>\vec \bold u </math> and <math>\vec \bold v </math> could be written as <math> \vec \bold u \cdot\vec \bold v = \sum_{k=1}^3 u_k v_k </math>.  If we follow our above analogy, we should be able to replace the sum over k with an integral over x.  There is one little notational problem, and that is we don't want to confuse the functional inner product with a simple muliply, so we need some new notation to denote this new inner product.  In [http://en.wikipedia.org/wiki/Quantum_mechanics quantum mechanics], physicists use the [http://en.wikipedia.org/wiki/Bra-ket_notation bra-ket] notation.  Let's borrow that.
 <math> <u|v> = \int_{-\infty}^\infty u^*(x) v(x) dx </math>
@@ Line 68: / Line 68: @@
 Note the complex conjugate on the function u(x).  That is in case u(x) is a complex valued function.  For the analogous case with vectors see [[Complex vector inner products]].
 ====Orthogonality for functions====
-Two functions, <math>u(t)</math> and <math>v(t)</math> are said to be orthogonal on the interval <math> (a,b) </math> with respect to the weighting function <math> w(t) </math>  if and only if
+Two functions, <math>u(t)\!</math> and <math>v(t)\!</math> are said to be orthogonal on the interval <math> (a,b) \!</math> with respect to the weighting function <math> w(t) \!</math>  if and only if
-<math>\int_a^b w(x) u^*(x) v(x) dx = 0 </math>.
+<math>\int_a^b w(x) u^*(x) v(x) dx = 0 \!</math>.
-The weighting function is often unity, but it is included so that different values of t can be weighted appropriately in analogy to the way the <math>w_k</math> weight was used when the vector basis set was orthogonal, but not orthonormal (that is, different basis vectors had different numerical lengths), as we discussed [[Vector weighting functions|here]].  Unless otherwise noted we will use <math> w(t) = 1 </math>, so that the defining relation for orthogonality of functions <math> u </math> and <math> v </math> becomes
+The weighting function is often unity, but it is included so that different values of <math>t\!</math> can be weighted appropriately in analogy to the way the <math>w_k\!</math> weight was used when the vector basis set was orthogonal, but not orthonormal (that is, different basis vectors had different numerical lengths), as we discussed [[Vector weighting functions|here]].  Unless otherwise noted we will use <math> w(t) = 1 \!</math>, so that the defining relation for orthogonality of functions <math> u \!</math> and <math> v \!</math> becomes
 <math>\int_a^b  u^*(x) v(x) dx = 0 </math>.
@@ Line 79: / Line 79: @@
 ==Other resources on orthogonality==
+[http://www.uvm.edu/~pdodds/files/papers/others/2000/sterian2000a.pdf Another professor's view of this topic...]
 [http://en.wikipedia.org/wiki/Inner_product Wikipedia inner product]
+[http://en.wikipedia.org/wiki/Orthogonal Wikipedia Orthogonality]
 Principle author of this page:  [[User:Frohro|Rob Frohne]]

Orthogonal functions: Difference between revisions

Latest revision as of 19:15, 28 September 2014

Contents

Introduction