Futurecraft Forums

A forum dedicated to communication and innovation!
 
HomeCalendarFAQSearchMemberlistUsergroupsRegisterLog in
Welcome, one and all, to the Futurecraft Forums!

Share | 
 

 Additional heightmap compression.

View previous topic View next topic Go down 
Go to page : Previous  1, 2, 3, 4, 5, 6, 7  Next
AuthorMessage
Ivan2006
General
General
avatar

Posts : 2096
Join date : 2012-05-08
Age : 19
Location : The Dungeon.

PostSubject: Re: Additional heightmap compression.   Fri Aug 23, 2013 3:50 am

^that.
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Fri Aug 23, 2013 3:57 am

I am an enabler.
Back to top Go down
View user profile
Keon
Lord/Lady Rear Admiral 1st
Lord/Lady Rear Admiral 1st
avatar

Posts : 3075
Join date : 2012-01-17
Location : Hahahaha.

PostSubject: Re: Additional heightmap compression.   Sat Aug 24, 2013 2:50 pm

So what is your goal with this thread? Heightmaps for distance viewing?

_________________
fr0stbyte124 wrote:
You know, I was thinking we should have a 3D crafting grid for complex recipes, but Kielaran is right: why have three dimensions when you can have six? Truly I don't know how we ever got by with a measly two.


Mackeroth? wrote:
Now, if Fr0stbyte wrote that, someone would make a sig, guaranteed. So what are you going to do now?
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Sat Aug 24, 2013 8:10 pm

Yes. And answering hypothetical nonsense.
Back to top Go down
View user profile
Iv121
General
General
avatar

Posts : 2396
Join date : 2012-02-05
Age : 22
Location : -> HERE ! <-

PostSubject: Re: Additional heightmap compression.   Sun Aug 25, 2013 12:11 am

After all

Code:
 
start: if [case1]
        {
            [case2]
        }
        else
        {
            goto start;
        }

Most useless if ever ! Razz
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Thu Sep 05, 2013 8:14 pm

Quick update, it would seem I didn't read the GPU Gems chapter on clipmaps thoroughly enough, because I misread a pretty major part.  The proxy mesh (pretty sure now that that is the official term for it) doesn't need the full NxN grid of vertices stored to memory.  Each ring is actually made up of square patches about 1/4 the span of the clip layer, and these are arranged in a ring surrounding a higher resolution instance in the center.  There are a couple other shapes related to how it is all stitched up, but that is the main element.  When you draw the mesh, you pass in scale and translation parameters, and if instancing is available, you can draw all the pieces with a single call.

The main thing, though, is that the memory footprint is smaller than I thought.

Now that I have the full algorithm worked out, I can probably implement it in about a day.  It'll look strange without the block shader in place, but it should be recognizable.  The main thing I want to get out of this is figuring out how far out the block shading makes a perceivable difference, and get a feel for how many ray steps will be needed to get an accurate fix.  If the really distant stuff is indistinguishable from smooth terrain, we might be able to shield ourselves from some of the worst-case scenarios.

The other thing I discovered is that the Intel GMA 950 doesn't actually support GLSL, at least in theory, which is a problem since it is a prerequisite even at the bare-minimum setup.  It does, however, support CG, which is NVidia's shader language from before HLSL and GLSL became the standard.  In fact, I believe both are actually compiled into CG on NVidia GPUs.  I'm a little fuzzy on that, but bottom line, it's possible I'll need to write a set of both if we're serious about not raising the minimum requirements.
Back to top Go down
View user profile
MercurySteam
Infantry
Infantry
avatar

Posts : 543
Join date : 2013-06-22

PostSubject: Re: Additional heightmap compression.   Thu Sep 05, 2013 9:53 pm

Yay?
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Thu Sep 05, 2013 9:59 pm

Your unbridled enthusiasm has been noted, and is appreciated.
Back to top Go down
View user profile
Ljubljana
Newbie
Newbie
avatar

Posts : 150
Join date : 2013-08-22
Age : 23
Location : I never leave

PostSubject: Re: Additional heightmap compression.   Fri Sep 06, 2013 12:06 am

So, is this actually going into code, or is it just an algorithm? And how are you/are you debugging it?
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Fri Sep 06, 2013 2:10 am

Algorithms go into code, yes.

I am currently implementing everything in C++ and using freeglut for the window manager.  Doing it native makes it easier to do certain things and gives a more reliable benchmark.  I'll port it over to LWJGL once I am happy with it.

Right now there is only a bare-bones amount of utility code. Everything is triggered via hotkeys, and there is no gui. I probably need to add one before too long.
Back to top Go down
View user profile
Ivan2006
General
General
avatar

Posts : 2096
Join date : 2012-05-08
Age : 19
Location : The Dungeon.

PostSubject: Re: Additional heightmap compression.   Fri Sep 06, 2013 3:50 am

Still sounds like progress.
Progress is good.
Back to top Go down
View user profile
Ljubljana
Newbie
Newbie
avatar

Posts : 150
Join date : 2013-08-22
Age : 23
Location : I never leave

PostSubject: Re: Additional heightmap compression.   Fri Sep 06, 2013 12:17 pm

Oh, I thought you were just making flowcharts or something, Thanks.
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Fri Sep 06, 2013 2:32 pm

Well I have a notebook full of those, too, so you're not wrong.  But this is implementation.  Right now I am running into some problems with ordering the vertices.  I've set it up so that the basic grid should be a set of 32x32 evenly spaced vertices, but the index ordering is still screwed up.  Because the grid is well-defined, I am trying to draw the triangles with the tri-strip primitive.

In openGL, there are a number of different primitives you can use to draw objects,



Points always use a single vertex, but you can vary the size of the rendered dot in the shader, which can be handy for particle systems.
Lines use two vertices, and line strips connect each vertex to the previous called one.  Line loops are the same as line strips, only the first and last vertex are also connected.
GL_Polygon is the filled-in version of line loops, and can theoretically make any shape, but it is extremely slow and nobody ever uses it.
GL_TRIANGLES specifies 3 vertices to generate a triangle.  This is the mode generally recommended unless you have a reason for using something else.
Triangle fan uses the first vertex as a pivot point, and every subsequent vertex links to the pivot and the previous vertex.  It can be handy for creating n-sided polygons, but is not frequently used in practice because you have to frequently restart new chains to create a whole mesh.
Triangle strip connects a vertex to the previous two vertices in the list.  If you can get a long chain, it is quite efficient, as you only need one new vertex per triangle.
GL_QUADS are what Minecraft uses.  Quads and quad strips are treated by the GPU as something akin to triangle strips, the only difference being that the diagonal doesn't show up in wireframe mode and you have a few extra options regarding flat shading.  The quad primitives have been removed from the standard as of OpenGL 3.1 (~DX10), so I want to avoid using them in my code.

When you draw primitives, you can input the vertices directly, but assuming a typical mesh will reuse a vertex multiple times, there is a better way.  Instead of inputing every vertex every time it is used, you put all your vertices into a buffer once, and then make a second buffer for indexing.  Each index references a vertex by its position in the vertex buffer, and considering that vertices are typically much larger than indexes (Minecraft vertices are 32 bytes apiece, while an index is at most 4), this saves a lot of space in GPU memory, but also has a second advantage, and that is vertex caching.

Once a vertex has been processed by the vertex shader, the results are temporarily cached in memory.  If that index appears again why the results are still in cache, you get the results for free.  If it's not in cache, the vertex shader has to reprocess it, so it is in your best interest to keep everything localized if you can help it.

What I am doing is a trick which optimizes vertex cache use by restarting each row before the vertex runs off the end of the cache.



The challenge here is to make sure that you don't go too long without resetting.  If it is even one too late, each vertex could end up leaving the cache right before it is called, resulting in every one of them being called twice, effectively halving the performance of this operation.  For modern GPUs, this is probably not a big deal, but this mesh is only generated once and used a lot, so I would prefer to make it as nice as possible before proceeding.

There is a tricky bit here because the vertex cache is first-in-first-out, meaning the ordering never changes.  To wield this efficiently, you need to pre-cache the entire initial row so that they leave the cache first.  You do this by creating degenerate triangles, which are zero-area triangles which reference one of their vertices twice.  The GPU recognizes these and skips generating a primitive for it, but since these generally mark the end of a triangle strip and the start of a new one, the vertices are still computed and cached.  If you con't pre-cache the first row, it will roughly half the length of a run because you will have two full rows in your cache at any given time, whereas pre-cached, you only need a single column of overlap between the first and second row.


Last edited by fr0stbyte124 on Fri Sep 06, 2013 2:54 pm; edited 3 times in total
Back to top Go down
View user profile
Ljubljana
Newbie
Newbie
avatar

Posts : 150
Join date : 2013-08-22
Age : 23
Location : I never leave

PostSubject: Re: Additional heightmap compression.   Fri Sep 06, 2013 2:45 pm

That... Actually makes sense, thanks!
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Fri Sep 06, 2013 11:07 pm

Yay, it worked!



Took way longer to work this out than I had hoped. Freaking scaling factors.
Back to top Go down
View user profile
ACH0225
General
General
avatar

Posts : 2346
Join date : 2012-01-01
Location : I might be somewhere, I might not.

PostSubject: Re: Additional heightmap compression.   Fri Sep 06, 2013 11:24 pm

Does this increase development by a percent? Or does it mean FC will come out as a mod before the novel and movie editions but after the RTS?
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Fri Sep 06, 2013 11:57 pm

Nah, this is nothing yet. But by the end of the weekend, I expect to have the full clipmap implemented and ready for experimentation.
Back to top Go down
View user profile
Iv121
General
General
avatar

Posts : 2396
Join date : 2012-02-05
Age : 22
Location : -> HERE ! <-

PostSubject: Re: Additional heightmap compression.   Sat Sep 07, 2013 2:11 am

Oh that wasn't even experimented upon yet -.-
Back to top Go down
View user profile
Ljubljana
Newbie
Newbie
avatar

Posts : 150
Join date : 2013-08-22
Age : 23
Location : I never leave

PostSubject: Re: Additional heightmap compression.   Sat Sep 07, 2013 1:34 pm

This is for really distant viewing, right?
Back to top Go down
View user profile
ACH0225
General
General
avatar

Posts : 2346
Join date : 2012-01-01
Location : I might be somewhere, I might not.

PostSubject: Re: Additional heightmap compression.   Sat Sep 07, 2013 1:38 pm

This is for FC: The Movie. It's going to help us with CGI.
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Mon Sep 09, 2013 5:05 am

Guh, this sucks.  I can't get anything to work.  Had no idea using mouse events was going to be such a big pain in the ass.


*EDIT*

Once again, it was my own damn fault.  The ordering of matrices in OpenGL is backwards from what I thought.  So you have a 4x4 matrix arranged as an array of 16 floating point elements.  The default arrangement is to complete all of column 1 before moving to column 2 and so on, rather than do all of row 1 before row 2.  This is pretty much opposite of every other matrix implementation I've seen.  But as it turns out, when you load the matrix into a shader, you can specify whether to transpose it first, which is essentially reversing the coordinate system.  I was using pieces from two different math libraries together, one of them relied on the transpose and the other was in native OpenGL format, so no matter how I moved the camera around, I wasn't finding any geometry.

That's what I get for trying to skip ahead and use things without learning how they work.

Camera class is almost done. Just getting the kinks out of the mouse events. I might switch to quaternion math eventually, but for now this will do.
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Tue Sep 10, 2013 2:05 pm

Just thought I would mention the camera is working now, and I just found out wikibooks has a bunch of OpenGL tutorials. http://en.wikibooks.org/wiki/OpenGL_Programming It's still pretty scattered, but what is there is pretty nice and covers a wide range of useful topics and modern libraries. There's even a set of tutorials on making a minecraft style engine, though I disagree with its use of the geometry shader. In theory, it could be useful for something like this, because you can turn a single point into a full cube of polygons, but that breaks down once you start wanting to add smooth lighting and baked-on ambient occlusion. Additionally, the geometry shader is a fatty, and doesn't parallelize nearly as well as the vertex or tessellation stages, so most developers avoid it like the plague unless they really need it for an effect, and even then it requires DX10 hardware, so you have to be prepared to go without anyway.
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Tue Sep 10, 2013 7:45 pm

I present: a planet



Or rather, part of one face of one, but this is to scale.  Right in the center, though it's hard to see, each of those smallest blocks are 32x32 in size, so basically, this is pretty big. Normally, you would always remain right in the center of this, but then it's harder to take a picture.

The cracks are there by design. Each octave has a gap which lets it shift around inside the next size hole. When we do hardware tessellation, I'll probably use a different approach which doesn't use sliding grids.

Next step making the grid shift with the camera and then reading the terrain off of texture maps, and then adjusting the elevation of the mesh based of the height field. Once that is done, it will look recognizably like terrain, though it won't be blocky at that point. However, that's where the fun begins and we can start testing the raycasting shader.


Last edited by fr0stbyte124 on Tue Sep 10, 2013 7:59 pm; edited 1 time in total
Back to top Go down
View user profile
Joel
Marine
Marine
avatar

Posts : 1473
Join date : 2012-04-01
Age : 20
Location : A Death World, stopping a Waaagh!

PostSubject: Re: Additional heightmap compression.   Tue Sep 10, 2013 7:50 pm

Glad to see something cool happening.
Back to top Go down
View user profile
fr0stbyte124
Super Developrator
Super Developrator
avatar

Posts : 1835
Join date : 2011-10-13

PostSubject: Re: Additional heightmap compression.   Wed Sep 11, 2013 1:54 am

Bam.

Spoiler:
 


Aaand Bam.

Spoiler:
 
Back to top Go down
View user profile
Sponsored content




PostSubject: Re: Additional heightmap compression.   

Back to top Go down
 
Additional heightmap compression.
View previous topic View next topic Back to top 
Page 2 of 7Go to page : Previous  1, 2, 3, 4, 5, 6, 7  Next
 Similar topics
-
» Details about ISO compression
» EX-70 1.4.2.0 compression modes question
» [ANSWERED] Inefficient ISO compression?
» How to do at home migraine device

Permissions in this forum:You cannot reply to topics in this forum
Futurecraft Forums :: Development :: Idea Center-
Jump to: