diff --git a/arbeit/bibma.bib b/arbeit/bibma.bib index b880b6a..9baf249 100644 --- a/arbeit/bibma.bib +++ b/arbeit/bibma.bib @@ -42,7 +42,6 @@ Url = {http://graphics.cs.brown.edu/~jfh/papers/Hsu-DMO-1992/paper.pdf} } - @Article{gaussNewton, Title = {An Algorithm for Least-Squares Estimation of Nonlinear Parameters}, Author = {Donald W. Marquardt}, @@ -184,3 +183,10 @@ url={https://pdfs.semanticscholar.org/a858/aa68da617ad9d41de021f6807cc422002258.pdf}, doi={10.1016/j.cagd.2012.03.025}, } +@article{brunet2010contributions, + title={Contributions to parametric image registration and 3d surface reconstruction}, + author={Brunet, Florent}, + journal={European Ph. D. in Computer Vision, Universit{\'e} dAuvergne, Cl{\'e}rmont-Ferrand, France, and Technische Universitat Munchen, Germany}, + year={2010}, + url={http://www.brnt.eu/phd/} +} diff --git a/arbeit/img/3dtarget.png b/arbeit/img/3dtarget.png new file mode 100644 index 0000000..26d6bc5 Binary files /dev/null and b/arbeit/img/3dtarget.png differ diff --git a/arbeit/img/ball.png b/arbeit/img/ball.png new file mode 100644 index 0000000..66bbac4 Binary files /dev/null and b/arbeit/img/ball.png differ diff --git a/arbeit/img/mario.png b/arbeit/img/mario.png new file mode 100644 index 0000000..0f1dc6b Binary files /dev/null and b/arbeit/img/mario.png differ diff --git a/arbeit/img/unity.png b/arbeit/img/unity.png new file mode 100644 index 0000000..2494849 Binary files /dev/null and b/arbeit/img/unity.png differ diff --git a/arbeit/ma.md b/arbeit/ma.md index 9380ac3..9f9eaea 100644 --- a/arbeit/ma.md +++ b/arbeit/ma.md @@ -186,12 +186,8 @@ convex--hull--property of B-Splines --- meaning, that no matter how we choose our coefficients, the resulting points all have to lie inside convex--hull of the control--points. -\improvement[inline]{ -Bilder von Basisfunktionen zur Visualisierung. -} - -For a given number of points $v_1,\dots,v_n$ we can then calculate -the contributions \linebreak[4]$n_{i,j}~:=~N_{j,d,\tau}$ of each control point $p_j$ to get the +For a given point $v_i$ we can then calculate the contributions +$n_{i,j}~:=~N_{j,d,\tau}$ of each control point $p_j$ to get the projection from the control--point--space into the object--space: $$ v_i = \sum_j n_{i,j} \cdot p_j = \vec{n}_i^{T} \vec{p} @@ -203,6 +199,32 @@ $$ where $\vec{N}$ is the $n \times m$ transformation--matrix (later on called **deformation matrix**) for $n$ object--space--points and $m$ control--points. +\begin{figure}[ht] +\begin{center} +\includegraphics[width=\textwidth]{img/unity.png} +\end{center} +\caption[B--spline--basis--function as partition of unity]{From \cite[Figure 2.13]{brunet2010contributions}:\newline +\glqq Some interesting properties of the B--splines. On the natural definition domain +of the B--spline ($[k_0,k_4]$ on this figure), the B--spline basis functions sum +up to one (partition of unity). In this example, we use B--splines of degree 2. +The horizontal segment below the abscissa axis represents the domain of +influence of the B--splines basis function, i.e. the interval on which they are +not null. At a given point, there are at most $ d+1$ non-zero B--spline basis +functions (compact support).\grqq \newline +Note, that Brunet starts his index at $-d$ opposed to our definition, where we +start at $0$.} +\label{fig:partition_unity} +\end{figure} + +Furthermore B--splines--basis--functions form a partition of unity for all, but +the first and last $d$ control-points\cite{brunet2010contributions}. Therefore +we later on use the border-points $d+1$ times, such that $\sum_j n_{i,j} p_j = p_i$ +for these points. + +The locality of the influence of each control--point and the partition of unity +was beautifully pictured by Brunet, which we included here as figure +\ref{fig:partition_unity}. + ### Why is \ac{FFD} a good deformation function? \label{sec:back:ffdgood} @@ -386,9 +408,9 @@ f(\vec{x} + \vec{U}\vec{p})$ this makes sure that $\|\vec{Up}\| \propto is only performed to map the criterion--range to $[0..1]$, whereas $1$ is the optimal value and $0$ is the worst value. -This criterion should be characteristic for numeric stability on the on -hand\cite[chapter 2.7]{golub2012matrix} and for convergence speed of -evolutionary algorithms on the other hand\cite{anrichterEvol} as it is tied to +On the one hand this criterion should be characteristic for numeric +stability\cite[chapter 2.7]{golub2012matrix} and on the other hand for the +convergence speed of evolutionary algorithms\cite{anrichterEvol} as it is tied to the notion of locality\cite{weise2012evolutionary,thorhauer2014locality}. ### Improvement Potential @@ -581,6 +603,8 @@ of the control--points.] to simulate different starting-conditions. # Scenarios for testing evolvability criteria using \acf{FFD} \label{sec:eval} +\improvement[inline]{für 1d und 3d entweder konsistent source/target oder +anders. Weil sonst in 1d $\vec{s}$ das Ziel, in 3d $\vec{t}$ das Ziel.} In our experiments we use the same two testing--scenarios, that were also used by \cite{anrichterEvol}. The first scenario deforms a plane into a shape @@ -594,15 +618,6 @@ point has three degrees of freedom in contrast to first scenario. ## Test Scenario: 1D Function Approximation -\begin{figure}[th] -\begin{center} -\includegraphics[width=0.7\textwidth]{img/1dtarget.png} -\end{center} -\caption{The target--shape for our 1--dimensional optimization--scenario -including a wireframe--overlay of the vertices.} -\label{fig:1dtarget} -\end{figure} - In this scenario we used the shape defined by Giannelli et al.\cite{giannelli2012thb}, which is also used by Richter et al.\cite{anrichterEvol} using the same discretization to $150 \times 150$ points for a total of $n = 22\,500$ vertices. The @@ -618,6 +633,15 @@ $$ with $(x,y) \in [0,2] \times [0,1]$ and $q(x,y)=(x-1.5)^2 + (y-0.5)^2$, which we have visualized in figure \ref{fig:1dtarget}. +begin{figure}[ht] +\begin{center} +\includegraphics[width=0.7\textwidth]{img/1dtarget.png} +\end{center} +\caption{The target--shape for our 1--dimensional optimization--scenario +including a wireframe--overlay of the vertices.} +\label{fig:1dtarget} +\end{figure} + As the starting-plane we used the same shape, but set all $z$--coordinates to $0$, yielding a flat plane, which is partially already correct. @@ -651,15 +675,6 @@ $$ where $\mathbb{1}$ is the vector consisting of $1$ in every dimension and $\vec{g}_\textrm{c} = \vec{p^{*}}$ the calculated correct gradient. -We then set up a regular 2--dimensional grid around the object with the desired -grid resolutions. To generate a testcase we then move the grid--vertices -randomly inside the x--y--plane. As we do not want to generate hard to solve -grids we avoid the generation of self--intersecting grids.\improvement{besser -formulieren} To achieve that we select a uniform distributed number -$r \in [-0.25,0.25]$ per dimension and shrink the distance to the neighbours -(the smaller neighbour for $r < 0$, the larger for $r > 0$) by the factor -$r$^[Note: On the Edges this displacement is only applied outwards by flipping the sign of $r$, if appropriate.]. - \begin{figure}[ht] \begin{center} \includegraphics[width=\textwidth]{img/example1d_grid.png} @@ -669,23 +684,89 @@ random distortion to generate a testcase.} \label{fig:example1d_grid} \end{figure} +We then set up a regular 2--dimensional grid around the object with the desired +grid resolutions. To generate a testcase we then move the grid--vertices +randomly inside the x--y--plane. As we do not want to generate hard to solve +grids we avoid the generation of self--intersecting grids.\improvement{besser +formulieren} To achieve that we select a uniform distributed number +$r \in [-0.25,0.25]$ per dimension and shrink the distance to the neighbours +(the smaller neighbour for $r < 0$, the larger for $r > 0$) by the factor +$r$^[Note: On the Edges this displacement is only applied outwards by flipping the sign of $r$, if appropriate.]. + An Example of such a testcase can be seen for a $7 \times 4$--grid in figure \ref{fig:example1d_grid}. ## Test Scenario: 3D Function Approximation -### Optimierungsszenario +Opposed to the 1--dimensional scenario before, the 3--dimensional scenario is +much more complex --- not only because we have more degrees of freedom on each +control point, but also because the *fitness--function* we will use has no known +analytic solution and multiple local minima. -- Ball zu Mario +\begin{figure}[ht] +\begin{center} +\includegraphics[width=0.7\textwidth]{img/3dtarget.png} +\end{center} +\caption{\newline +Left: The sphere we start from with 10\,807 vertices\newline +Right: The face we want to deform the sphere into with 12\,024 vertices.} +\label{fig:3dtarget} +\end{figure} -### Matching in 3D +First of all we introduce the set up: We have given a triangulated model of a +sphere consisting of 10\,807 vertices, that we want to deform into a +the target--model of a face with a total of 12\,024 vertices. Both of +these Models can be seen in figure \ref{fig:3dtarget}. -- alternierende Optimierung +Opposed to the 1D--case we cannot map the source and target--vertices in a +one--to--one--correspondence, which we especially need for the approximation of +the fitting--error. Hence we state that the error of one vertex is the distance +to the closest vertex of the other model. -### Besonderheiten der Optimierung +We therefore define the *fitness--function* to be: +$$ +f(\vec{P}) = \frac{1}{n} \underbrace{\sum_{i=1}^n \|\vec{c_T(s_i)} - +\vec{s_i}\|_2^2}_{\textrm{source-to-target--distance}} ++ \frac{1}{m} \underbrace{\sum_{i=1}^m \|\vec{c_S(t_i)} - +\vec{t_i}\|_2^2}_{\textrm{target-to-source--distance}} ++ \lambda \cdot \textrm{regularization} +$$ +where $\vec{c_T(s_i)}$ denotes the target--vertex that is corresponding to the +source--vertex $\vec{s_i}$ and $\vec{c_S(t_i)}$ denotes the source--vertex that +corresponds to the target--vertex $\vec{t_i}$. Note that the target--vertices +are given and fixed by the target--model of the face we want to deform into, +whereas the source--vertices vary depending on the chosen parameters $\vec{P}$, +as those get calculated by the previously introduces formula $\vec{S} = \vec{UP}$ +with $\vec{S}$ being the $n \times 3$--matrix of source--vertices, $\vec{U}$ the +$n \times m$--matrix of calculated coefficients for the \ac{FFD} --- analog to +the 1D case --- and finally $\vec{P}$ being the $m \times 3$--matrix of the +control--grid defining the whole deformation. -- Analytische Lösung nur bis zur Optimierung der ersten Punkte gültig -- Kriterien trotzdem gut +As regularization-term we introduce a weighted decaying +laplace--coefficient\unsure{heisst der so?} +that is known to speed up the optimization--process\improvement{cite [34] aus +ref{anrichterEvol}} and simulates a material that is very stiff in the +beginning --- to do a coarse deformation --- and gets easier to deform over +time. + +\improvement[inline]{mehr zu regularisierung, Formel etc.} + +## Procedure: 3D Function Approximation + +Initially we set up the correspondences $\vec{c_T(\dots)}$ and $\vec{c_S(\dots)}$ to be +the respectively closest vertices of the other model. We then calculate the +analytical solution given these correspondences via $\vec{P^{*}} = \vec{U^+}\vec{T}$, +and also use the first solution as guessed gradient for the calculation of the +*improvement--potential*, as the optimal solution is not known. +We then let the evolutionary algorithm run up within $1.05$ times the error of +this solution and afterwards recalculate the correspondences $\vec{c_T(\dots)}$ +and $\vec{c_S(\dots)}$. +For the next step we then halve the regularization--impact $\lambda$ and +calculate the next incremental solution $\vec{P^{*}} = \vec{U^+}\vec{T}$ with +the updated correspondences to get our next target--error. +We repeat this process as long as the target--error keeps decreasing. + +\improvement[inline]{grid-setup} # Evaluation of Scenarios \label{sec:res} @@ -716,9 +797,13 @@ An Example of such a testcase can be seen for a $7 \times 4$--grid in figure \begin{figure}[!ht] -\includegraphics[width=\textwidth]{img/evolution3d/20171007_3dFit_all_append.png} -\caption{Results 3D} +\includegraphics[width=\textwidth]{img/evolution3d/4x4xX_montage.png} +\caption{Results 3D for 4x4xX} +\end{figure} +\begin{figure}[!ht] +\includegraphics[width=\textwidth]{img/evolution3d/Xx4x4_montage.png} +\caption{Results 3D for Xx4x4} \end{figure} diff --git a/arbeit/ma.pdf b/arbeit/ma.pdf index 7fdb776..0ed9e83 100644 Binary files a/arbeit/ma.pdf and b/arbeit/ma.pdf differ diff --git a/arbeit/ma.tex b/arbeit/ma.tex index e901af3..f5446fd 100644 --- a/arbeit/ma.tex +++ b/arbeit/ma.tex @@ -339,14 +339,9 @@ to the convex--hull--property of B-Splines --- meaning, that no matter how we choose our coefficients, the resulting points all have to lie inside convex--hull of the control--points. -\improvement[inline]{ -Bilder von Basisfunktionen zur Visualisierung. -} - -For a given number of points \(v_1,\dots,v_n\) we can then calculate the -contributions \linebreak[4]\(n_{i,j}~:=~N_{j,d,\tau}\) of each control -point \(p_j\) to get the projection from the control--point--space into -the object--space: \[ +For a given point \(v_i\) we can then calculate the contributions +\(n_{i,j}~:=~N_{j,d,\tau}\) of each control point \(p_j\) to get the +projection from the control--point--space into the object--space: \[ v_i = \sum_j n_{i,j} \cdot p_j = \vec{n}_i^{T} \vec{p} \] or written for all points at the same time: \[ \vec{v} = \vec{N} \vec{p} @@ -354,6 +349,33 @@ v_i = \sum_j n_{i,j} \cdot p_j = \vec{n}_i^{T} \vec{p} on called \textbf{deformation matrix}) for \(n\) object--space--points and \(m\) control--points. +\begin{figure}[ht] +\begin{center} +\includegraphics[width=\textwidth]{img/unity.png} +\end{center} +\caption[B--spline--basis--function as partition of unity]{From \cite[Figure 2.13]{brunet2010contributions}:\newline +\glqq Some interesting properties of the B--splines. On the natural definition domain +of the B--spline ($[k_0,k_4]$ on this figure), the B--spline basis functions sum +up to one (partition of unity). In this example, we use B--splines of degree 2. +The horizontal segment below the abscissa axis represents the domain of +influence of the B--splines basis function, i.e. the interval on which they are +not null. At a given point, there are at most $ d+1$ non-zero B--spline basis +functions (compact support).\grqq \newline +Note, that Brunet starts his index at $-d$ opposed to our definition, where we +start at $0$.} +\label{fig:partition_unity} +\end{figure} + +Furthermore B--splines--basis--functions form a partition of unity for +all, but the first and last \(d\) +control-points\cite{brunet2010contributions}. Therefore we later on use +the border-points \(d+1\) times, such that \(\sum_j n_{i,j} p_j = p_i\) +for these points. + +The locality of the influence of each control--point and the partition +of unity was beautifully pictured by Brunet, which we included here as +figure \ref{fig:partition_unity}. + \subsection{\texorpdfstring{Why is \ac{FFD} a good deformation function?}{Why is a good deformation function?}}\label{why-is-a-good-deformation-function} @@ -561,10 +583,10 @@ The inversion of \(\kappa(\vec{U})\) is only performed to map the criterion--range to \([0..1]\), whereas \(1\) is the optimal value and \(0\) is the worst value. -This criterion should be characteristic for numeric stability on the on -hand\cite[chapter 2.7]{golub2012matrix} and for convergence speed of -evolutionary algorithms on the other hand\cite{anrichterEvol} as it is -tied to the notion of +On the one hand this criterion should be characteristic for numeric +stability\cite[chapter 2.7]{golub2012matrix} and on the other hand for +the convergence speed of evolutionary algorithms\cite{anrichterEvol} as +it is tied to the notion of locality\cite{weise2012evolutionary,thorhauer2014locality}. \subsection{Improvement Potential}\label{improvement-potential} @@ -764,6 +786,8 @@ using \acf{FFD}}{Scenarios for testing evolvability criteria using }}\label{scenarios-for-testing-evolvability-criteria-using} \label{sec:eval} +\improvement[inline]{für 1d und 3d entweder konsistent source/target oder +anders. Weil sonst in 1d $\vec{s}$ das Ziel, in 3d $\vec{t}$ das Ziel.} In our experiments we use the same two testing--scenarios, that were also used by \cite{anrichterEvol}. The first scenario deforms a plane @@ -779,15 +803,6 @@ scenario. \section{Test Scenario: 1D Function Approximation}\label{test-scenario-1d-function-approximation} -\begin{figure}[th] -\begin{center} -\includegraphics[width=0.7\textwidth]{img/1dtarget.png} -\end{center} -\caption{The target--shape for our 1--dimensional optimization--scenario -including a wireframe--overlay of the vertices.} -\label{fig:1dtarget} -\end{figure} - In this scenario we used the shape defined by Giannelli et al.\cite{giannelli2012thb}, which is also used by Richter et al.\cite{anrichterEvol} using the same discretization to @@ -803,6 +818,16 @@ s(x,y) = \(q(x,y)=(x-1.5)^2 + (y-0.5)^2\), which we have visualized in figure \ref{fig:1dtarget}. +begin\{figure\}{[}ht{]} + +\begin{center} +\includegraphics[width=0.7\textwidth]{img/1dtarget.png} +\end{center}\caption{The target--shape for our 1--dimensional optimization--scenario +including a wireframe--overlay of the vertices.} +\label{fig:1dtarget} + +\textbackslash{}end\{figure\} + As the starting-plane we used the same shape, but set all \(z\)--coordinates to \(0\), yielding a flat plane, which is partially already correct. @@ -839,6 +864,15 @@ consider a distorted gradient \(\vec{g}_{\textrm{d}}\) \[ dimension and \(\vec{g}_\textrm{c} = \vec{p^{*}}\) the calculated correct gradient. +\begin{figure}[ht] +\begin{center} +\includegraphics[width=\textwidth]{img/example1d_grid.png} +\end{center} +\caption{\newline Left: A regular $7 \times 4$--grid\newline Right: The same grid after a +random distortion to generate a testcase.} +\label{fig:example1d_grid} +\end{figure} + We then set up a regular 2--dimensional grid around the object with the desired grid resolutions. To generate a testcase we then move the grid--vertices randomly inside the x--y--plane. As we do not want to @@ -851,47 +885,84 @@ neighbours (the smaller neighbour for \(r < 0\), the larger for displacement is only applied outwards by flipping the sign of \(r\), if appropriate.}. -\begin{figure}[ht] -\begin{center} -\includegraphics[width=\textwidth]{img/example1d_grid.png} -\end{center} -\caption{\newline Left: A regular $7 \times 4$--grid\newline Right: The same grid after a -random distortion to generate a testcase.} -\label{fig:example1d_grid} -\end{figure} - An Example of such a testcase can be seen for a \(7 \times 4\)--grid in figure \ref{fig:example1d_grid}. \section{Test Scenario: 3D Function Approximation}\label{test-scenario-3d-function-approximation} -\subsection{Optimierungsszenario}\label{optimierungsszenario} +Opposed to the 1--dimensional scenario before, the 3--dimensional +scenario is much more complex --- not only because we have more degrees +of freedom on each control point, but also because the +\emph{fitness--function} we will use has no known analytic solution and +multiple local minima. -\begin{itemize} -\tightlist -\item - Ball zu Mario -\end{itemize} +\begin{figure}[ht] +\begin{center} +\includegraphics[width=0.7\textwidth]{img/3dtarget.png} +\end{center} +\caption{\newline +Left: The sphere we start from with 10\,807 vertices\newline +Right: The face we want to deform the sphere into with 12\,024 vertices.} +\label{fig:3dtarget} +\end{figure} -\subsection{Matching in 3D}\label{matching-in-3d} +First of all we introduce the set up: We have given a triangulated model +of a sphere consisting of 10,807 vertices, that we want to deform into a +the target--model of a face with a total of 12,024 vertices. Both of +these Models can be seen in figure \ref{fig:3dtarget}. -\begin{itemize} -\tightlist -\item - alternierende Optimierung -\end{itemize} +Opposed to the 1D--case we cannot map the source and target--vertices in +a one--to--one--correspondence, which we especially need for the +approximation of the fitting--error. Hence we state that the error of +one vertex is the distance to the closest vertex of the other model. -\subsection{Besonderheiten der -Optimierung}\label{besonderheiten-der-optimierung} +We therefore define the \emph{fitness--function} to be: \[ +f(\vec{P}) = \frac{1}{n} \underbrace{\sum_{i=1}^n \|\vec{c_T(s_i)} - +\vec{s_i}\|_2^2}_{\textrm{source-to-target--distance}} ++ \frac{1}{m} \underbrace{\sum_{i=1}^m \|\vec{c_S(t_i)} - +\vec{t_i}\|_2^2}_{\textrm{target-to-source--distance}} ++ \lambda \cdot \textrm{regularization} +\] where \(\vec{c_T(s_i)}\) denotes the target--vertex that is +corresponding to the source--vertex \(\vec{s_i}\) and \(\vec{c_S(t_i)}\) +denotes the source--vertex that corresponds to the target--vertex +\(\vec{t_i}\). Note that the target--vertices are given and fixed by the +target--model of the face we want to deform into, whereas the +source--vertices vary depending on the chosen parameters \(\vec{P}\), as +those get calculated by the previously introduces formula +\(\vec{S} = \vec{UP}\) with \(\vec{S}\) being the \(n \times 3\)--matrix +of source--vertices, \(\vec{U}\) the \(n \times m\)--matrix of +calculated coefficients for the \ac{FFD} --- analog to the 1D case --- +and finally \(\vec{P}\) being the \(m \times 3\)--matrix of the +control--grid defining the whole deformation. -\begin{itemize} -\tightlist -\item - Analytische Lösung nur bis zur Optimierung der ersten Punkte gültig -\item - Kriterien trotzdem gut -\end{itemize} +As regularization-term we introduce a weighted decaying +laplace--coefficient\unsure{heisst der so?} that is known to speed up +the optimization--process\improvement{cite [34] aus +ref{anrichterEvol}} and simulates a material that is very stiff in the +beginning --- to do a coarse deformation --- and gets easier to deform +over time. + +\improvement[inline]{mehr zu regularisierung, Formel etc.} + +\section{Procedure: 3D Function +Approximation}\label{procedure-3d-function-approximation} + +Initially we set up the correspondences \(\vec{c_T(\dots)}\) and +\(\vec{c_S(\dots)}\) to be the respectively closest vertices of the +other model. We then calculate the analytical solution given these +correspondences via \(\vec{P^{*}} = \vec{U^+}\vec{T}\), and also use the +first solution as guessed gradient for the calculation of the +\emph{improvement--potential}, as the optimal solution is not known. We +then let the evolutionary algorithm run up within \(1.05\) times the +error of this solution and afterwards recalculate the correspondences +\(\vec{c_T(\dots)}\) and \(\vec{c_S(\dots)}\). For the next step we then +halve the regularization--impact \(\lambda\) and calculate the next +incremental solution \(\vec{P^{*}} = \vec{U^+}\vec{T}\) with the updated +correspondences to get our next target--error. We repeat this process as +long as the target--error keeps decreasing. + +\improvement[inline]{grid-setup} \chapter{Evaluation of Scenarios}\label{evaluation-of-scenarios} @@ -926,9 +997,13 @@ Approximation}\label{results-of-1d-function-approximation} Approximation}\label{results-of-3d-function-approximation} \begin{figure}[!ht] -\includegraphics[width=\textwidth]{img/evolution3d/20171007_3dFit_all_append.png} -\caption{Results 3D} +\includegraphics[width=\textwidth]{img/evolution3d/4x4xX_montage.png} +\caption{Results 3D for 4x4xX} +\end{figure} +\begin{figure}[!ht] +\includegraphics[width=\textwidth]{img/evolution3d/Xx4x4_montage.png} +\caption{Results 3D for Xx4x4} \end{figure} \chapter{Schluss}\label{schluss}