constrained graphic layout generation via latent optimization

Constrained Graphic Layout Generation via Latent Optimization Paper reading notes _ Don't ask about the endless blog of future learning -CSDN Blog The task is based on the labels and constraints of the layout elements , Generate high-quality and reasonable layout (Layout). In fact, predecessors have done a lot of work to deal with this problem , however , The previous work is to directly generate the layout , But when generating the layout , There may be various occlusions , Overlap, etc , Therefore, after designing a new neural network to solve the problem of layout generation , Using the idea of optimization , The objective function and limiting conditions are designed , Optimize the final generation effect . Methods the whole method is divided into two Part, As above, pipeline Shown are LayoutGan++ and Constrained Layout Generation via Late...https://blog.csdn.net/baidu_40582876/article/details/121637804Constrained Graphic Layout Generation via Latent Optimization(2021)_studyeboy The blog of -CSDN Blog [Paper] Constrained Graphic Layout Generation via Latent Optimization[Code] ktrk115/const_layout Generate constrained graphic layout summaries through potential optimization in graphic design , It is common for humans to visually arrange various elements according to design intent and semantics . for example , The title text almost always appears at the top of other elements in the document . In this work , The graphic layout we generate can flexibly combine this design semantics , Whether implicitly or explicitly specified by the user . We use the potential space of the ready-made layout generation model for optimization , Allow our approach to match existing https://blog.csdn.net/studyeboy/article/details/122861414 This article is intuitive and difficult to understand ,

1.Introduction

        Avoid alignment or overlap by additional loss or adjustment , One disadvantage of integrating constraints into learning goals is , When users want to merge new bundles , The model must adapt to new conditions or new losses , contrary , We choose to perform optimization in the potential space of the generated model , Allow the use of existing models .

        Constrained layout Generation via Latent Optimization(CLG-LO), Constrained layout generation through potential space optimization , It defines layout generation as a constrained optimization problem in potential space , As shown in the figure below , Generated using unconstrained settings and specifications GAN The model acts as a constrained optimization network , Use iterative algorithm to optimize the potential space of unconstrained model , A layout that meets the specified constraints has been found . Allow users to use a single pre training network GAN, And merge various constraints into layout generation as required . This is the core summary of this article , Single training layoutgan++, Then based on Lagrange function optimization , Add constraints to the potential space to find the optimal layout .

2.related work

2.1 layout generation

        Classical optimization methods manually design energy functions with a large number of constraints , The layout should meet these constraints , For example, Microsoft's Automatic Generation of Visual-Textual Presentation Layout, Very complicated , Want to reproduce this framework , It costs a lot . In addition to graphic design layout , There is also indoor scene layout design , Are all problems in this field .constrained layout generation At the same time, unconstrained generation and constrained generation are considered , be based on layoutgan Build an unconstrained layout builder , And user layout constraints are applied to the learning generator .

2.2 latent space exploitation

        latent z If it is image synthesis , It's a probability distribution , Mainstream research involves projecting target images into potential space , And use user input to perform image editing in the learned manifold . For example, some image restoration and coloring tasks .

3.approach

        The goal is to generate a semantically reasonable and high-quality design layout from a set of element labels and constraints specified by the user . First train a layoutgan++, An unconstrained layout generation model , The model is then used to constrain the generation task .

3.1 layoutgan++

generator ： use transformer In the form of , instead of layoutgan Complex in self-attention, The code is very concise .

Judging device ： The author uses layoutgan The wireframe renderer of has done a comparative experiment , It is found that the effect has not improved , In particular, the wireframe renderer becomes unstable when the data set size is limited . Here the author is layoutgan Of git It is also mentioned in issue.

auxiliary decoder： Experience has found that , In well aligned layouts such as documents , The discriminator is trained to be sensitive to alignment and less sensitive to location information , That is, it only cares about whether the elements are aligned , And don't care about the unusual layout , For example, put the title element at the bottom , Therefore, in order to let the discriminator know the location information , Additional regularization is applied , Add an auxiliary decoder to rebuild the bounding box .

training objective:

Here the author trained a and layoutgan The same upgraded version layoutgan++, Yes layoutgan Wireframe renderer in ,self-attention And so on , The unconstrained layout generation model becomes more concise , If only here , Also no problem , However, the subsequent layout generation with embedded constraints is the focus of the author .

3.2 Constrained layout generation via latent optimizer(CLG-LO)

        Using the idea of optimization , Add constraints to hidden space , Find an optimal hidden space to ensure that the generated results meet such constraints . Update parameters through iteration .