Futurecraft Forums

A forum dedicated to communication and innovation!
 
HomeCalendarFAQSearchMemberlistUsergroupsRegisterLog in
Welcome, one and all, to the Futurecraft Forums!

Share | 
 

 Additional heightmap compression.

View previous topic View next topic Go down 
Go to page : Previous  1, 2, 3, 4, 5, 6, 7  Next
AuthorMessage
Iv121
General
General
avatar

Posts : 2396
Join date : 2012-02-05
Age : 22
Location : -> HERE ! <-

PostSubject: Re: Additional heightmap compression.   Wed Sep 11, 2013 2:41 am

For me the real question is how would it look from space, and if the answer if crap all of that is not worth for me.
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Wed Sep 11, 2013 3:39 am

Here is roughly orbit distance.

Keep in mind that this is bare-bones specs atm, and it will be possible to give each band a different resolution texture.  I'm thinking if the 1st and 2n outer bands had the same resolution as the 3rd band (at most 834 KB more), that would make for a satisfying minimum resolution.

Spoiler:
 

I'll add the adjustment tomorrow.
Back to top Go down
View user profile
Iv121
General
General
avatar

Posts : 2396
Join date : 2012-02-05
Age : 22
Location : -> HERE ! <-

PostSubject: Re: Additional heightmap compression.   Wed Sep 11, 2013 4:03 am

And then you get a whole new task on your hand - to make a less chaotic terrain generator ...
Back to top Go down
View user profile
Ivan2006
General
General
avatar

Posts : 2096
Join date : 2012-05-08
Age : 19
Location : The Dungeon.

PostSubject: Re: Additional heightmap compression.   Wed Sep 11, 2013 8:56 am

Iv121 wrote:
And then you get a whole new task on your hand - to make a less chaotic terrain generator ...
^that
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Wed Sep 11, 2013 12:47 pm

Well this was strange. I tried to run the program on my work computer and it crashed trying to allocate the texture memory.
It's an AMD Radeon HD 6450, and should be supported up OpenGL 4.1 (DX11), and I am targeting OpenGL 3.0 and what it crashed on in core functionality. And there's no way it could have run out of memory, it just...didn't work.

I'm updating the drivers. If that doesn't fix this may be a problem.
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Wed Sep 11, 2013 1:32 pm

I have no idea.  I switched the OpenGL context from 3.0 to 3.1 and it worked.  3.2 doesn't.  This is so annoying.

*Edit*
And now it's working again. It might have to do with the shader version, I'm not sure.
Back to top Go down
View user profile
Ivan2006
General
General
avatar

Posts : 2096
Join date : 2012-05-08
Age : 19
Location : The Dungeon.

PostSubject: Re: Additional heightmap compression.   Wed Sep 11, 2013 2:27 pm

So your super-advanced rendering engine VS basic version error?
Well, I heard all the fancy programming abilities in the world don´t make it easier to fix those...
Back to top Go down
View user profile
Groot
Marine
Marine
avatar

Posts : 1456
Join date : 2012-03-18
Age : 21
Location : Yggdrasil

PostSubject: Re: Additional heightmap compression.   Wed Sep 11, 2013 2:32 pm

Is this the good old decrease the poly count as you get further away?
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Wed Sep 11, 2013 2:51 pm

Ivan2006 wrote:
So your super-advanced rendering engine VS basic version error?
Well, I heard all the fancy programming abilities in the world don´t make it easier to fix those...
This is more a case of this thing has no excuse for not working but still inexplicably doesn't.  We saw this in Starmade, too.  The Intel HD4000 should have been able to support the advanced shader profile, but refused.  I don't get it.

Prototype wrote:
Is this the good old decrease the poly count as you get further away?
Sure is.  Once the shaders are in, it will look like a block world again, but this is the true geometry.

Once all the heightmap stuff is to my liking, there's a couple other models I need to integrate to make it fully volumetric, but this will be the meat of the far draw system.
Back to top Go down
View user profile
Delta
Sergeant
Sergeant
avatar

Posts : 903
Join date : 2012-03-26
Age : 19
Location : Virginia

PostSubject: Re: Additional heightmap compression.   Wed Sep 11, 2013 3:38 pm

fr0stbyte124 wrote:
Ivan2006 wrote:
So your super-advanced rendering engine VS basic version error?
Well, I heard all the fancy programming abilities in the world don´t make it easier to fix those...
This is more a case of this thing has no excuse for not working but still inexplicably doesn't.  We saw this in Starmade, too.  The Intel HD4000 should have been able to support the advanced shader profile, but refused.  I don't get it.
So we have issues with integrated GPUs... Could this be Java's fault?
Back to top Go down
View user profile http://arsln100.forumotion.com
Ljubljana
Newbie
Newbie
avatar

Posts : 150
Join date : 2013-08-22
Age : 23
Location : I never leave

PostSubject: Re: Additional heightmap compression.   Wed Sep 11, 2013 3:52 pm

Hory shet, progress!
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Wed Sep 11, 2013 4:42 pm

Solar112 wrote:
fr0stbyte124 wrote:
Ivan2006 wrote:
So your super-advanced rendering engine VS basic version error?
Well, I heard all the fancy programming abilities in the world don´t make it easier to fix those...
This is more a case of this thing has no excuse for not working but still inexplicably doesn't.  We saw this in Starmade, too.  The Intel HD4000 should have been able to support the advanced shader profile, but refused.  I don't get it.
So we have issues with integrated GPUs... Could this be Java's fault?
Nope, I'm in C++ right now.  I'll keep experimenting.


Anyway, did another test, this time I just pumped in the full-res terrain for coloring.  The geometry is still the same as before.
Spoiler:
 
I'm not sure what mip levels are being used here but it clearly makes a huge difference.  It looks fantastic at ground level as well.
Also note that the variations in elevation are barely noticeable at this scale.  You can see a bit of it poking around the cracks, but for the most part, it is indistinguishable from flat.  With some basic normal mapping (the ridges already there were baked on by World Painter), I would say this geometry resolution is plenty.
Back to top Go down
View user profile
Ljubljana
Newbie
Newbie
avatar

Posts : 150
Join date : 2013-08-22
Age : 23
Location : I never leave

PostSubject: Re: Additional heightmap compression.   Wed Sep 11, 2013 4:50 pm

So is this going to be the default world size? Seems a bit small to me, but I'm the kind of person who wants a breathing mechanic to be implemented, so that's irrelevant.
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Wed Sep 11, 2013 7:13 pm

Smaller, actually.

It's hard to get a sense of the scale without any atmospheric cues, but the surface of that particular map is 67 km^2, or twice the size of the world in GTA San Andreas.  For one face.  Stored regularly on disk, it's 1.2 GB of world data.  We'll be cutting about 1.5km off of each edge and converting it to atmosphere, so you can have adequate build height.

Incidentally, the new height cap puts you roughly at low orbit, so space elevators will be possible.
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Wed Sep 11, 2013 7:53 pm

One more quick update, just tested the mip level.  At ground level, the maximum useful resolution is 1px/m out to 1km (closer to 800, really), 1px /2m up to 2km, 1px /4m up to 3km, and  1px/8m out to 4km, and so on.  That means 4 complete tiers of 1024x1024, which, assuming only color and DXT1 compression, is ~2MB best case.  For colored side faces, 4MB.  Well within even a conservative budget.

That doesn't include any geometry or height fields, but I'm still working out how much detail is actually needed to reproduce those.
Back to top Go down
View user profile
MercurySteam
Infantry
Infantry
avatar

Posts : 543
Join date : 2013-06-22

PostSubject: Re: Additional heightmap compression.   Wed Sep 11, 2013 11:07 pm

This is exciting. I am excited.
Back to top Go down
View user profile
Joel
Marine
Marine
avatar

Posts : 1473
Join date : 2012-04-01
Age : 20
Location : A Death World, stopping a Waaagh!

PostSubject: Re: Additional heightmap compression.   Wed Sep 11, 2013 11:28 pm


If you couldn't already tell, I am excited as well.
Spock's excited too.
Back to top Go down
View user profile
MercurySteam
Infantry
Infantry
avatar

Posts : 543
Join date : 2013-06-22

PostSubject: Re: Additional heightmap compression.   Thu Sep 12, 2013 12:34 am



Yes. This is truly a momentous occasion.
Back to top Go down
View user profile
Keon
Lord/Lady Rear Admiral 1st
Lord/Lady Rear Admiral 1st
avatar

Posts : 3075
Join date : 2012-01-17
Location : Hahahaha.

PostSubject: Re: Additional heightmap compression.   Thu Sep 12, 2013 9:35 am

Too lazy to find a picard facepalm face.

Anyways, I'd advise oct-trees. Or octaves. Always the answer.

Also



This guy's managed to solve a bunch of our problems here.

_________________
fr0stbyte124 wrote:
You know, I was thinking we should have a 3D crafting grid for complex recipes, but Kielaran is right: why have three dimensions when you can have six? Truly I don't know how we ever got by with a measly two.


Mackeroth? wrote:
Now, if Fr0stbyte wrote that, someone would make a sig, guaranteed. So what are you going to do now?
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Thu Sep 12, 2013 10:50 am

Shocked  He is the hero Gotham deserves.


This is really cool.  I'm not crazy about the continued dependence on polygon chunking, but I think he's on the right path.  As a matter of fact, you might even be able to fake cubes on this terrain with our shader.  Right now it only works with heightfields, but it looks like approximation off of normal maps is going to play a bigger role than I planned.

The true voxel far terrain looks to be marching cubes or a variation, based on the detail popping.  And I'm loving the tree imposters.  I wonder how that system works, and why the interior is more consistent than the edges? My guess is the GPU is given a region in which to populate the forest, and the GPU is just using roughly the same generation algorithm as the CPU. In other words, the forest parameters need to be known ahead of time.
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Thu Sep 12, 2013 12:07 pm

Looked at more videos, to do the round planets, they start with a cube and deform it to a sphere.  The far view volumetric structure isn't grid based so it doesn't care, and the grid chunks work by abusing what I am going to call warp points, so chunks will line up on different sides depending on the direction you approach it from.  This makes all the world flat (though some really visible shifting in certain places) but the 8 cube corners are non-traversable, because chunks would meet up at 120 degree angles.

It's nice to see at least the attempt, but I still don't want to go down that route for FC.  I think the distortion model will cause more and more trouble for us as the project progresses, whereas the cube model gets all of its trouble out in the open from the very start.


------------------------------
I'm working on texture storage today.  I had some pics up with an 8192x8192 texture, but using the full thing uncompressed is 192 MB, which is ridiculous.  I got the world texture converted to DXT1 using photoshop, and the accuracy is amazingly good, though I am a little concerned we won't be able to use raw colors until the half-res layer.    I'll keep it this way for now, though.

*edit*
Crap I made a mistake.  I said the range of 1 block per pixel was ~1024, so really full coverage ought to be a 2048x2048 map since you are at the center.  That means the memory footprint will be 4x what I thought.  Alternatively, we could accept that we're losing detail too soon and switch to half res at 512m.  That's still 2x the max distance of vanilla MC, so maybe it's enough.  This might actually be plenty after we add atmospheric scattering or a slight depth-of-field blur.

For now, I'm going to go with that as the assumption.  We can always crank it up if it's not enough.
Back to top Go down
View user profile
Ljubljana
Newbie
Newbie
avatar

Posts : 150
Join date : 2013-08-22
Age : 23
Location : I never leave

PostSubject: Re: Additional heightmap compression.   Thu Sep 12, 2013 12:31 pm

fr0stbyte124 wrote:
That's still 2x the max distance of vanilla MC
That's about the max distance of optifine, right? That would be plenty far, assuming I understand you correctly.
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Thu Sep 12, 2013 3:07 pm

Ljubljana wrote:
fr0stbyte124 wrote:
That's still 2x the max distance of vanilla MC
That's about the max distance of optifine, right? That would be plenty far, assuming I understand you correctly.
It will probably be fine. I'm just trying to make sure I do this right because it's a chore to change these resolution because they affect so many other structures.

-------------------------------
Another point of interest:

I'm still not certain on the best encoding of the height fields. Right now, it is using an 8 bit height component, but that won't be enough if we want awesome mountains and stuff. The normal solution would be to add a second lower resolution height field across the entire world that would take care of the low frequency variation and paint with a wider brush, so to speak, but if this layer is interpolated, which would be sensible for natural terrain, then getting the high frequency adjustments to match perfectly with integer values could be difficult, and if the terrain changes enough to affect the average on the low frequency height field, you have to regenerate a large section of the high frequency map to compensate.

Alternatively, you could read in the low frequency height field with Nearest Neighbor, which results in Minecraft-y discrete textures. In this way, you will have more well-defined heights. Say we had a low frequency range of 0-4096, so anything from the center to the ceiling. Storing it to an 8-bit height field could be done using increments of 16. Then, if the high frequency height field is 8-bit, all the heights can deviate by ± 127. If the low frequency heightfield is a 256x256 texure, each block would contain a 32x32 region, so in theory you could have a maximum slope of roughly 82 degrees. I think that will be sufficient.

A third options would be to have the base height passed into the instanced mesh patches like its global offset is now, saving a texture unit. However, it would be constant over the entire patch, and the patches can scale up to 2048x2048, so that's probably not an option. The texture should be fine.
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Thu Sep 12, 2013 4:05 pm

Yay, more low-priority mysteries are being solved!  I figured out why my screen space scaling was off.  The way my projection matrix is set up, you specify the field of view, but for the Y axis, not the X axis.  I've been using 60 degrees, which for a 16:9 screen means the horizontal FOV is actually 91.5 degrees.

Additionally, the distance a 1m block needs to be to take up exactly 1 px is Z = yRes / (2*tan(yFOV / 2)), so at 1080p, the cutoff is 935m.  At 512m, the block is still 1.82 px in size.
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Thu Sep 12, 2013 8:09 pm

Well I'm bad at math. I forgot I was doing powers of 2 for mip resolution.

For max quality:

Full res: < 935m
1/2: < 1871m
1/4: < 3741m
1/8: < 7482m
1/16: < 14,964m

And so on. Incidentally, a planet will take up at least 1px up to 4,675 km away.

What this means for us is that using a clipmap of 2048x2048 would require 3 full levels to cover the whole planet.
DXT1 for two of the layers is 8MB. If the first level uses 2 bytes for indexing, that might be another 8MB. Not too bad, but none of that is the heightfields.

If we map to 1024x1024 and accept that it will switch before 1px resolution single pixel (which isn't necessarily bad, either), then we need 4 full levels.
DXT1 for 3 of them is 3MB. Indexing on first one is 2MB. Let's say another 170 KB for the remaining mips.

16 bit indexing could be encoded as r5g6b5, so 3 channels of 32,64,32 values. The way I imagine this working is you have up to 64 blockIDs indexed as the top block type, and 32 indexed for the blocks below it. You can have 1-32 of these blocks, and the rest will be rendered stone. If the actual block is not indexed, this can be patched over on a per-tile basis afterward.

The DXT colors are straight color values. Since they are mostly for sub-pixel blocks, they have no need for more than one color, and this will save some texture lookups. Each column will have two color values: one for the top and one for the side.
Back to top Go down
View user profile
Sponsored content




PostSubject: Re: Additional heightmap compression.   

Back to top Go down
 
Additional heightmap compression.
View previous topic View next topic Back to top 
Page 3 of 7Go to page : Previous  1, 2, 3, 4, 5, 6, 7  Next
 Similar topics
-
» Details about ISO compression
» [ANSWERED] EX-70 1.4.2.0 compression modes question
» [ANSWERED] Inefficient ISO compression?
» How to do at home migraine device

Permissions in this forum:You cannot reply to topics in this forum
Futurecraft Forums :: Development :: Idea Center-
Jump to: