Multi-layer text proj update

I took Not-so-early version and added concat and projection from multiple text encoder layers, initialized to 0 0 0 1 at start (identical to default), and let it train for 10 hours. Layers that were used: 7, 14, 21, 28. It would appear that model really wanted deeper conditioning, as it converged onto next values:
7: 27.7%
14: 18.4%
21: 2.9%
28: 51.0% (down from 100%)

That would suggest that optimal setup would be something like an exponential, L3, 7, 14, 28, or something like that. Model did not take almost any new conditioning from layers close to output, but happily added almost relative half of it from deep layers.

This small projector added 8mb of weights, and you'll need a node to use this model properly. No other changes done - https://github.com/Anzhc/Anima-Multi-Layer-Text-Projection

Things i can note:

It did force some of forgetting, as conditioning is adjusted. Main knowledge, like big characters, is not affected, but less known characters are, or i think so.

Reduced style bias from the base.

Backgrounds started to make a bit more sense, and have more details on average now.

P.S. There could be some other issues im not aware of.

Older descriptions below

Nothing interesting going on here, move along.

Just a small finetune of Anima, 4 epochs with 12k images. Nothing interesting was done. Is more 2.5d, will lead towards more abstract, that's about it.

Not-so-early upd: another 5 epochs of 12k images with nl. Then 25 epochs of subset of 430 images with nl written by codex. So total of ~72k samples on top of 48k, for a total of 120k.

Issues

~~Can have strong bokeh in some cases.~~

~~Some wide shots are transforming into less common compositions for which there is barely any data and denoise fails hard on them.~~
Idk, seem fixed.

Positives

Tends to add more details.

Elements in the scene usually interact more (i.e. abstract features affecting more concepts)

Some complex compositions become better defined.

Booba is bigger on average. (But i will not benchmark that(for now))

Less common concepts with abstract meaning may perform better (like horror)

Other Features

On average, less overwhelming contrast, dimmer.

Can use higher cfg without blowing out colors (i use 4-6, vs 3.5 on base), but not required.

Params i use

dpm++ sde, 16-24 steps, cfg 3.5-7, shift 4

How was it trained

Lr: variable LR per block group, from 3e-6 to 5e-6, from early to late. Additional modification for LR scaling based on bucket resolution
Adapter: Trained for not-so-early, 5e-7.
Loss: L2, with some modifications for rare aspect ratio buckets and depth.
Schedule: Logit-Normal -0.2 1.5(or was it 1.5 -0.2...)(Bluvoll's schedule) with shift 4 and normal tail sampling modification. (Tldr, all that fixes lack of front in default logit normal and normalizes amount of sampling at the very late timesteps, without making them overbearing at high shift.)

Captions: ~~tags only.~~ ~~(you won't make me suffer through captioning 12k images with gemma 4 31b or qwen 3.6 27b locally for NL counterpart)~~
Tags+NL, i suffered for 5 days.