diff --git a/arbeit/bibma.bib b/arbeit/bibma.bib index 9baf249..b314dce 100644 --- a/arbeit/bibma.bib +++ b/arbeit/bibma.bib @@ -190,3 +190,15 @@ year={2010}, url={http://www.brnt.eu/phd/} } +@article{aschenbach2015, + author = {Achenbach, Jascha and Zell, Eduard and Botsch, Mario}, + booktitle = {Vision, Modeling \& Visualization}, + journal = {Proceedings of Vision, Modeling and Visualization}, + location = {Aachen, Germany}, + pages = {1--8}, + publisher = {Eurographics Association}, + title = {Accurate Face Reconstruction through Anisotropic Fitting and Eye Correction}, + year = {2015}, + url = {http://graphics.uni-bielefeld.de/publications/disclaimer.php?dlurl=vmv15.pdf}, + ISBN = {978-3-905674-95-8}, +} diff --git a/arbeit/img/example3d_grid.png b/arbeit/img/example3d_grid.png new file mode 100644 index 0000000..db1eb8d Binary files /dev/null and b/arbeit/img/example3d_grid.png differ diff --git a/arbeit/ma.md b/arbeit/ma.md index 9f9eaea..c0f72da 100644 --- a/arbeit/ma.md +++ b/arbeit/ma.md @@ -548,7 +548,6 @@ $$J(Err(u,v,w)) \cdot \Delta \left( \begin{array}{c} u \\ v \\ w \end{array} \ri and use Cramers rule for inverting the small Jacobian and solving this system of linear equations. - ## Deformation Grid As mentioned in chapter \ref{sec:back:evo}, the way of choosing the @@ -594,7 +593,7 @@ central points are not relevant for the parametrization of the circle. \unsure[inline]{erwähnen, dass man aus $\vec{D}$ einfach die Null--Spalten entfernen kann?} -For our tests we chose different uniformly sized grids and added gaussian noise +For our tests we chose different uniformly sized grids and added noise onto each control-point^[For the special case of the outer layer we only applied noise away from the object, so the object is still confined in the convex hull of the control--points.] to simulate different starting-conditions. @@ -603,8 +602,6 @@ of the control--points.] to simulate different starting-conditions. # Scenarios for testing evolvability criteria using \acf{FFD} \label{sec:eval} -\improvement[inline]{für 1d und 3d entweder konsistent source/target oder -anders. Weil sonst in 1d $\vec{s}$ das Ziel, in 3d $\vec{t}$ das Ziel.} In our experiments we use the same two testing--scenarios, that were also used by \cite{anrichterEvol}. The first scenario deforms a plane into a shape @@ -623,7 +620,7 @@ which is also used by Richter et al.\cite{anrichterEvol} using the same discretization to $150 \times 150$ points for a total of $n = 22\,500$ vertices. The shape is given by the following definition $$ -s(x,y) = +t(x,y) = \begin{cases} 0.5 \cos(4\pi \cdot q^{0.5}) + 0.5 & q(x,y) < \frac{1}{16},\\ 2(y-x) & 0 < y-x < 0.5,\\ @@ -633,7 +630,7 @@ $$ with $(x,y) \in [0,2] \times [0,1]$ and $q(x,y)=(x-1.5)^2 + (y-0.5)^2$, which we have visualized in figure \ref{fig:1dtarget}. -begin{figure}[ht] +\begin{figure}[ht] \begin{center} \includegraphics[width=0.7\textwidth]{img/1dtarget.png} \end{center} @@ -649,9 +646,9 @@ correct. Regarding the *fitness--function* $f(\vec{p})$, we use the very simple approach of calculating the squared distances for each corresponding vertex $$ -\textrm{f(\vec{p})} = \sum_{i=1}^{n} \|(\vec{Up})_i - s_i\|_2^2 = \|\vec{Up} - \vec{s}\|^2 \rightarrow \min +\textrm{f(\vec{p})} = \sum_{i=1}^{n} \|(\vec{Up})_i - t_i\|_2^2 = \|\vec{Up} - \vec{t}\|^2 \rightarrow \min $$ -where $s_i$ are the respective solution--vertices to the parametrized +where $t_i$ are the respective target--vertices to the parametrized source--vertices^[The parametrization is encoded in $\vec{U}$ and the initial position of the control points. See \ref{sec:ffd:adapt}] with the current deformation--parameters $\vec{p} = (p_1,\dots, p_m)$. We can do this @@ -659,11 +656,11 @@ one--to--one--correspondence because we have exactly the same number of source and target-vertices do to our setup of just flattening the object. This formula is also the least--squares approximation error for which we -can compute the analytic solution $\vec{p^{*}} = \vec{U^+}\vec{s}$, yielding us +can compute the analytic solution $\vec{p^{*}} = \vec{U^+}\vec{t}$, yielding us the correct gradient in which the evolutionary optimizer should move. ## Procedure: 1D Function Approximation - + For our setup we first compute the coefficients of the deformation--matrix and use then the formulas for *variability* and *regularity* to get our predictions. Afterwards we solve the problem analytically to get the (normalized) correct @@ -679,19 +676,26 @@ $\vec{g}_\textrm{c} = \vec{p^{*}}$ the calculated correct gradient. \begin{center} \includegraphics[width=\textwidth]{img/example1d_grid.png} \end{center} -\caption{\newline Left: A regular $7 \times 4$--grid\newline Right: The same grid after a +\caption[Example of a 1D--grid]{\newline Left: A regular $7 \times 4$--grid\newline Right: The same grid after a random distortion to generate a testcase.} \label{fig:example1d_grid} \end{figure} We then set up a regular 2--dimensional grid around the object with the desired grid resolutions. To generate a testcase we then move the grid--vertices -randomly inside the x--y--plane. As we do not want to generate hard to solve -grids we avoid the generation of self--intersecting grids.\improvement{besser -formulieren} To achieve that we select a uniform distributed number -$r \in [-0.25,0.25]$ per dimension and shrink the distance to the neighbours -(the smaller neighbour for $r < 0$, the larger for $r > 0$) by the factor -$r$^[Note: On the Edges this displacement is only applied outwards by flipping the sign of $r$, if appropriate.]. +randomly inside the x--y--plane. As self-intersecting grids get tricky to solve +with our implemented newtons--method we avoid the generation of such +self--intersecting grids for our testcases. + +This is a reasonable thing to do, as self-intersecting grids violate our desired +property of locality, as the then farther away control--point has more influence +over some vertices as the next-closer. + +To achieve that we select a uniform distributed number $r \in [-0.25,0.25]$ per +dimension and shrink the distance to the neighbours (the smaller neighbour for +$r < 0$, the larger for $r > 0$) by the factor $r$^[Note: On the Edges this +displacement is only applied outwards by flipping the sign of $r$, if +appropriate.]. An Example of such a testcase can be seen for a $7 \times 4$--grid in figure \ref{fig:example1d_grid}. @@ -729,7 +733,7 @@ f(\vec{P}) = \frac{1}{n} \underbrace{\sum_{i=1}^n \|\vec{c_T(s_i)} - \vec{s_i}\|_2^2}_{\textrm{source-to-target--distance}} + \frac{1}{m} \underbrace{\sum_{i=1}^m \|\vec{c_S(t_i)} - \vec{t_i}\|_2^2}_{\textrm{target-to-source--distance}} -+ \lambda \cdot \textrm{regularization} ++ \lambda \cdot \textrm{regularization}(\vec{P}) $$ where $\vec{c_T(s_i)}$ denotes the target--vertex that is corresponding to the source--vertex $\vec{s_i}$ and $\vec{c_S(t_i)}$ denotes the source--vertex that @@ -742,14 +746,21 @@ $n \times m$--matrix of calculated coefficients for the \ac{FFD} --- analog to the 1D case --- and finally $\vec{P}$ being the $m \times 3$--matrix of the control--grid defining the whole deformation. -As regularization-term we introduce a weighted decaying -laplace--coefficient\unsure{heisst der so?} -that is known to speed up the optimization--process\improvement{cite [34] aus -ref{anrichterEvol}} and simulates a material that is very stiff in the -beginning --- to do a coarse deformation --- and gets easier to deform over -time. +As regularization-term we add a weighted Laplacian of the deformation that has +been used before by Aschenbach et al.\cite[Section 3.2]{aschenbach2015} on +similar models and was shown to lead to a more precise fit. The Laplacian +$$ +\textrm{regularization}(\vec{P}) = \frac{1}{\sum_i A_i} \sum_{i=1}^n A_i \cdot \left( \sum_{\vec{s_j} \in \mathcal{N}(\vec{s_i})} w_j \cdot \|\Delta \vec{s_j} - \Delta \vec{\overline{s}_j}\|^2 \right) +$$ +is determined by the cotangent weighted displacement $w_j$ of the to $s_i$ +connected vertices $\mathcal{N}(s_i)$ and $A_i$ is the Voronoi--area of the corresponding vertex +$\vec{s_i}$. We leave out the $\vec{R}_i$--term from the original paper as our +deformation is merely linear. -\improvement[inline]{mehr zu regularisierung, Formel etc.} +This regularization--weight gives us a measure of stiffness for the material +that we will influence via the $\lambda$--coefficient to start out with a stiff +material that will get more flexible per iteration. +\unsure[inline]{Andreas: hast du nen cite, wo gezeigt ist, dass das so sinnvoll ist?} ## Procedure: 3D Function Approximation @@ -766,7 +777,31 @@ calculate the next incremental solution $\vec{P^{*}} = \vec{U^+}\vec{T}$ with the updated correspondences to get our next target--error. We repeat this process as long as the target--error keeps decreasing. -\improvement[inline]{grid-setup} +\begin{figure}[ht] +\begin{center} +\includegraphics[width=\textwidth]{img/example3d_grid.png} +\end{center} +\caption[Example of a 3D--grid]{\newline Left: The 3D--setup with a $4\times +4\times 4$--grid.\newline Right: The same grid after added noise to the +control--points.} +\label{fig:setup3d} +\end{figure} + +The grid we use for our experiments is just very coarse due to computational +limitations. We are not interested in a good reconstruction, but an estimate if +the mentioned evolvability criteria are good. + +In figure \ref{fig:setup3d} we show an example setup of the scene with a +$4\times 4\times 4$--grid. Identical to the 1--dimensional scenario before, we create a +regular grid and move the control-points uniformly random between their +neighbours, but in three instead of two dimensions^[Again, we flip the signs for +the edges, if necessary to have the object still in the convex hull.]. + +As is clearly visible from figure \ref{fig:3dtarget}, the target--model has many +vertices in the facial area, at the ears and in the neck--region. Therefore we +chose to increase the grid-resolutions for our tests in two different dimensions +and see how well the criteria predict a suboptimal placement of these +control-points. # Evaluation of Scenarios \label{sec:res} @@ -817,3 +852,6 @@ We repeat this process as long as the target--error keeps decreasing. \label{sec:dis} HAHA .. als ob -.- + +\improvement[inline]{Bibliotheksverzeichnis links anpassen. DOI überschreibt +Direktlinks des Autors.} diff --git a/arbeit/ma.pdf b/arbeit/ma.pdf index 0ed9e83..1bce5cd 100644 Binary files a/arbeit/ma.pdf and b/arbeit/ma.pdf differ diff --git a/arbeit/ma.tex b/arbeit/ma.tex index f5446fd..9808c91 100644 --- a/arbeit/ma.tex +++ b/arbeit/ma.tex @@ -773,11 +773,11 @@ circle. \unsure[inline]{erwähnen, dass man aus $\vec{D}$ einfach die Null--Spalten entfernen kann?} -For our tests we chose different uniformly sized grids and added -gaussian noise onto each control-point\footnote{For the special case of - the outer layer we only applied noise away from the object, so the - object is still confined in the convex hull of the control--points.} -to simulate different starting-conditions. +For our tests we chose different uniformly sized grids and added noise +onto each control-point\footnote{For the special case of the outer layer + we only applied noise away from the object, so the object is still + confined in the convex hull of the control--points.} to simulate +different starting-conditions. \unsure[inline]{verweis auf DM--FFD?} @@ -786,8 +786,6 @@ using \acf{FFD}}{Scenarios for testing evolvability criteria using }}\label{scenarios-for-testing-evolvability-criteria-using} \label{sec:eval} -\improvement[inline]{für 1d und 3d entweder konsistent source/target oder -anders. Weil sonst in 1d $\vec{s}$ das Ziel, in 3d $\vec{t}$ das Ziel.} In our experiments we use the same two testing--scenarios, that were also used by \cite{anrichterEvol}. The first scenario deforms a plane @@ -808,7 +806,7 @@ al.\cite{giannelli2012thb}, which is also used by Richter et al.\cite{anrichterEvol} using the same discretization to \(150 \times 150\) points for a total of \(n = 22\,500\) vertices. The shape is given by the following definition \[ -s(x,y) = +t(x,y) = \begin{cases} 0.5 \cos(4\pi \cdot q^{0.5}) + 0.5 & q(x,y) < \frac{1}{16},\\ 2(y-x) & 0 < y-x < 0.5,\\ @@ -818,15 +816,14 @@ s(x,y) = \(q(x,y)=(x-1.5)^2 + (y-0.5)^2\), which we have visualized in figure \ref{fig:1dtarget}. -begin\{figure\}{[}ht{]} - +\begin{figure}[ht] \begin{center} \includegraphics[width=0.7\textwidth]{img/1dtarget.png} -\end{center}\caption{The target--shape for our 1--dimensional optimization--scenario +\end{center} +\caption{The target--shape for our 1--dimensional optimization--scenario including a wireframe--overlay of the vertices.} \label{fig:1dtarget} - -\textbackslash{}end\{figure\} +\end{figure} As the starting-plane we used the same shape, but set all \(z\)--coordinates to \(0\), yielding a flat plane, which is partially @@ -835,10 +832,10 @@ already correct. Regarding the \emph{fitness--function} \(f(\vec{p})\), we use the very simple approach of calculating the squared distances for each corresponding vertex \[ -\textrm{f(\vec{p})} = \sum_{i=1}^{n} \|(\vec{Up})_i - s_i\|_2^2 = \|\vec{Up} - \vec{s}\|^2 \rightarrow \min -\] where \(s_i\) are the respective solution--vertices to the -parametrized source--vertices\footnote{The parametrization is encoded in - \(\vec{U}\) and the initial position of the control points. See +\textrm{f(\vec{p})} = \sum_{i=1}^{n} \|(\vec{Up})_i - t_i\|_2^2 = \|\vec{Up} - \vec{t}\|^2 \rightarrow \min +\] where \(t_i\) are the respective target--vertices to the parametrized +source--vertices\footnote{The parametrization is encoded in \(\vec{U}\) + and the initial position of the control points. See \ref{sec:ffd:adapt}} with the current deformation--parameters \(\vec{p} = (p_1,\dots, p_m)\). We can do this one--to--one--correspondence because we have exactly the same number of @@ -846,7 +843,7 @@ source and target-vertices do to our setup of just flattening the object. This formula is also the least--squares approximation error for which we -can compute the analytic solution \(\vec{p^{*}} = \vec{U^+}\vec{s}\), +can compute the analytic solution \(\vec{p^{*}} = \vec{U^+}\vec{t}\), yielding us the correct gradient in which the evolutionary optimizer should move. @@ -868,17 +865,22 @@ correct gradient. \begin{center} \includegraphics[width=\textwidth]{img/example1d_grid.png} \end{center} -\caption{\newline Left: A regular $7 \times 4$--grid\newline Right: The same grid after a +\caption[Example of a 1D--grid]{\newline Left: A regular $7 \times 4$--grid\newline Right: The same grid after a random distortion to generate a testcase.} \label{fig:example1d_grid} \end{figure} We then set up a regular 2--dimensional grid around the object with the desired grid resolutions. To generate a testcase we then move the -grid--vertices randomly inside the x--y--plane. As we do not want to -generate hard to solve grids we avoid the generation of -self--intersecting grids.\improvement{besser -formulieren} To achieve that we select a uniform distributed number +grid--vertices randomly inside the x--y--plane. As self-intersecting +grids get tricky to solve with our implemented newtons--method we avoid +the generation of such self--intersecting grids for our testcases. + +This is a reasonable thing to do, as self-intersecting grids violate our +desired property of locality, as the then farther away control--point +has more influence over some vertices as the next-closer. + +To achieve that we select a uniform distributed number \(r \in [-0.25,0.25]\) per dimension and shrink the distance to the neighbours (the smaller neighbour for \(r < 0\), the larger for \(r > 0\)) by the factor \(r\)\footnote{Note: On the Edges this @@ -922,7 +924,7 @@ f(\vec{P}) = \frac{1}{n} \underbrace{\sum_{i=1}^n \|\vec{c_T(s_i)} - \vec{s_i}\|_2^2}_{\textrm{source-to-target--distance}} + \frac{1}{m} \underbrace{\sum_{i=1}^m \|\vec{c_S(t_i)} - \vec{t_i}\|_2^2}_{\textrm{target-to-source--distance}} -+ \lambda \cdot \textrm{regularization} ++ \lambda \cdot \textrm{regularization}(\vec{P}) \] where \(\vec{c_T(s_i)}\) denotes the target--vertex that is corresponding to the source--vertex \(\vec{s_i}\) and \(\vec{c_S(t_i)}\) denotes the source--vertex that corresponds to the target--vertex @@ -936,14 +938,22 @@ calculated coefficients for the \ac{FFD} --- analog to the 1D case --- and finally \(\vec{P}\) being the \(m \times 3\)--matrix of the control--grid defining the whole deformation. -As regularization-term we introduce a weighted decaying -laplace--coefficient\unsure{heisst der so?} that is known to speed up -the optimization--process\improvement{cite [34] aus -ref{anrichterEvol}} and simulates a material that is very stiff in the -beginning --- to do a coarse deformation --- and gets easier to deform -over time. +As regularization-term we add a weighted Laplacian of the deformation +that has been used before by Aschenbach et +al.\cite[Section 3.2]{aschenbach2015} on similar models and was shown to +lead to a more precise fit. The Laplacian \[ +\textrm{regularization}(\vec{P}) = \frac{1}{\sum_i A_i} \sum_{i=1}^n A_i \cdot \left( \sum_{\vec{s_j} \in \mathcal{N}(\vec{s_i})} w_j \cdot \|\Delta \vec{s_j} - \Delta \vec{\overline{s}_j}\|^2 \right) +\] is determined by the cotangent weighted displacement \(w_j\) of the +to \(s_i\) connected vertices \(\mathcal{N}(s_i)\) and \(A_i\) is the +Voronoi--area of the corresponding vertex \(\vec{s_i}\). We leave out +the \(\vec{R}_i\)--term from the original paper as our deformation is +merely linear. -\improvement[inline]{mehr zu regularisierung, Formel etc.} +This regularization--weight gives us a measure of stiffness for the +material that we will influence via the \(\lambda\)--coefficient to +start out with a stiff material that will get more flexible per +iteration. +\unsure[inline]{Andreas: hast du nen cite, wo gezeigt ist, dass das so sinnvoll ist?} \section{Procedure: 3D Function Approximation}\label{procedure-3d-function-approximation} @@ -962,7 +972,33 @@ incremental solution \(\vec{P^{*}} = \vec{U^+}\vec{T}\) with the updated correspondences to get our next target--error. We repeat this process as long as the target--error keeps decreasing. -\improvement[inline]{grid-setup} +\begin{figure}[ht] +\begin{center} +\includegraphics[width=\textwidth]{img/example3d_grid.png} +\end{center} +\caption[Example of a 3D--grid]{\newline Left: The 3D--setup with a $4\times +4\times 4$--grid.\newline Right: The same grid after added noise to the +control--points.} +\label{fig:setup3d} +\end{figure} + +The grid we use for our experiments is just very coarse due to +computational limitations. We are not interested in a good +reconstruction, but an estimate if the mentioned evolvability criteria +are good. + +In figure \ref{fig:setup3d} we show an example setup of the scene with a +\(4\times 4\times 4\)--grid. Identical to the 1--dimensional scenario +before, we create a regular grid and move the control-points uniformly +random between their neighbours, but in three instead of two +dimensions\footnote{Again, we flip the signs for the edges, if necessary + to have the object still in the convex hull.}. + +As is clearly visible from figure \ref{fig:3dtarget}, the target--model +has many vertices in the facial area, at the ears and in the +neck--region. Therefore we chose to increase the grid-resolutions for +our tests in two different dimensions and see how well the criteria +predict a suboptimal placement of these control-points. \chapter{Evaluation of Scenarios}\label{evaluation-of-scenarios} @@ -1012,6 +1048,9 @@ Approximation}\label{results-of-3d-function-approximation} HAHA .. als ob -.- +\improvement[inline]{Bibliotheksverzeichnis links anpassen. DOI überschreibt +Direktlinks des Autors.} + % \backmatter \cleardoublepage