Discord 
TUTORIAL VIDEO: OjO made a great tutorial to create photogrammetry sceneries with the MSFS decoder, and my optimization tools. Feel free to have a look here: https://youtu.be/6fgTxEh_x84
SIMULATION UPDATE 5: Thanks to the update, photogrammetry sceneries use far less RAM than before, and should perform better (even if they were already optimized for performances).
IMPORTANT: for the merge_sceneries script to work properly, the source scenery and the destination scenery must share the same min LOD (17 here). The reason is that the min LOD determines the size of the tiles, and the size of the tiles from the source scenery must be the same as the one from the destination scenery.
Python blender scripts designed to be used with Google Earth Decoder Tool sceneries (https://github.com/Thalixte/Google-Earth-Decoder-optimisation-tools).
1) Presentation:
The archive contains four python scripts:
- scenery_optimisation.py: optimize Google Earth Decoder scenery (textures, Lods, CTD fix)
- fix_tiles_altitudes.py: fix wrong tiles altitudes that can occure when moving tiles on the x-y axis
- update_objects_LODs.py: update the LOD levels for the tiles of a scenery based on an array of minsizes
- merge_sceneries: merge all the files from a source Google Earth Decoder scenery to a destination Google Earth Decoder scenery
The scripts are intended to be used with a Google Earth Decoder min LOD of 17. I did not test them with another Google Earth Decoder minLod.
2) Prerequisites:
- Blender 2.83 (https://download.blender.org/release/Blender2.83/blender-2.83.9-windows64.msi). The blender version is important
- Node js (https://nodejs.org/dist/v14.15.1/node-v14.15.1-x64.msi)
- Blender2MSFS Toolkit: https://www.fsdeveloper.com/forum/resources/blender2msfs-toolkit.256/download
- Lily texture Packer IMPORTANT ! download version 1.1.x only: https://gumroad.com/l/DFExj
3) scenery_optimisation script:
3.1) Presentation:
This script applies the CTD issues fix (see https://flightsim.to/blog/creators-guide-fix-ctd-issues-on-your-scenery).
It bakes the tiles texture files in order to reduce the number of the files in the final package, and reduce the scenery loading time.
It fixes an issue with the Google Earth Decoder tool, that breaks the LOD management system.
Look at this picture: https://flic.kr/p/2k3YCrd
This picture shows a wireframe representation of a big scenery created from the Google Earth Decoder tool. You can see that the scenery tiles are dark gray, which means there are a lot of vertices on each tile, because all the tiles are on the max level of detail (which is 19 here). The result is that this scene, from this point of view, displays more than 80M of vertices, which bottlenecks the GPU (and the CPU).
Normally, we should have this: https://flic.kr/p/2kayM52. In this case, we can see that the tiles have the correct LOD levels, and it leads to a reduced number of vertices (20M in this case, but it can be changed according to the LOD levels that are set in the optimisation script (see chap. 3.3 - Configuration)).
The problem here is that the Google Earth Decoder gives all the tiles the same origin, and changes the bounding box to go from this origin point to the last mesh vertice point.
Another way to see this problem, is to use the new MSFS SDK Debug LOD feature, that displays the current lod level of each tiles, and the size of the bounding sphere: https://flic.kr/p/2kautrR
The more the green or blue is the color, the more detailed is the tile. The more the yellow or red is the color, the less detailed is the tile.
Here, we can see that the more detailed tiles are the one that are far from the camera, which is the exact opposite result than the one we want to obtain to optimize our scenery framerate.
Now, if each tile has its own origin point, and has a bounding box corresponding to the real tile size, we obtain those results: https://flic.kr/p/2kayiyp and https://flic.kr/p/2kayiAo, which are far better from a LOD point of view, and better preserve the framerate.
This is the fix that the script applies to the tiles: it gives each tile its own origin point, and resizes the bounding boxes according to the real tile size.
The script also changes the LOD levels to better suit the LOD management system, and allows (if configured) to convert texture files into jpg format, in order to reduce the texture file size (but with a possible loss in the texture quality).
The script also applies ASOBO extension tags to the gltf files, in order to enable the road management and the collisions. It also fixes texture flickering issues.
The script automatically removes the orphaned scenery object xml files (scenery object xml files that do not have associated gltf and/or bin files)
3.2) Installation:
Just put the scenery_optimisation.py script, and the retrievepos.js script in a folder of your choice.
3.3) Configuration:
Change the following settings, according to your project:
- bake_textures_enabled: tells the script to optimize the textures by baking all the textures corresponding to the min Lod levels of the tiles (default is True). For instance, if you have a gltf file for a tile that is named 30604141705340627_LOD00.gltf, all the texture files corresponding to this tile and this LOD level (all the texture files that start with 30604141705340627_LOD00) will be baked into one single texture
- projects_folder: the parent folder that contains your sceneries
- project_name: the name of your project
- node_js_folder: the folder that contains the node js script that retrieves the Google Earth coords
- fspackagetool_folder: the folder that contains the fspackagetool exe that builds the MSFS packages
- target_lods: an array representing the minsize values per LOD, starting from a minLod of 17 (from the less detailed lod to the most detailed)
- project_file_name: the name of the xml file that embeds the project definition (by default, project_name.xml or author_name+project_name.xml)
- scene_file_name: the name of the xml file that embeds the tile descriptions (by default, objects.xml)
- package_definitions_file_name: the name of the xml file that embeds the package definitions (by default, project_name.xml or author_name+project_name.xml)
- author_name: the name of the author of the scenery
- build_package_enabled: enable the package compilation when the script has finished (default is True)
- output_texture_format: format of the final texture files (values are PNG_FORMAT, JPG_FORMAT, default is PNG_FORMAT)
- JPG_COMPRESSION_RATIO: if you choose the jpg format for the output texture files, indicates the compression ratio
3.4) Usage:
Open Blender in administrator mode.
Go in the Scripting view, then click on the Open icon, and choose the scenery_optimisation python script.
When the configuration is done, open the Blender system console Window (Window => Toggle System Console). Then, run the script.
3.5) Process:
- rename the modelLib folder, in order to fix CTD issues (see https://flightsim.to/blog/creators-guide-fix-ctd-issues-on-your-scenery/)
- if no other backup exists, backup the scenery modelLib files into a backup folder, inside the project folder
- if necessary, install the node Js xhr2 module (npm install xhr2)
- install the PIP and Pillow libraries to allow texture format conversion
- if png texture files output format is selected, convert all remaining jpg texture files to PNG, then remove jpg files
- if jpg texture files output format is selected, convert all remaining png texture files to JPG, then remove png files, and compress all jpg texture files
- retrieve objects positions from Google earth, via the Google Earth API, and put the results in .pos files corresponding to the scenery tiles
- update the tiles position, using those .pos files
- place all tiles objects into corresponding sub folders, in order to group the objects corresponding to the same tile
- update the LODs of the scenery tiles, according to the target_lods defined in the configuration settings
- optimize the tiles, by baking the textures corresponding to LOD levels of the tiles, and by changing the bounding box of the tiles, in order to optimize the LODs
- applies ASOBO extension tags to the gltf files, in order to enable the road management and the collisions
- fix gltf doublesided attributes, in order to remove texture flickering issues
- removes the orphaned scenery object xml files (scenery object xml files that do not have associated gltf and/or bin files)
- automatically rebuild the scenery package, if the MSFS SDK is correctly installed, and MSFS 2020 is not running
4) fix_tiles_altitudes script :
4.1) Presentation:
This script applies the CTD issues fix (see https://flightsim.to/blog/creators-guide-fix-ctd-issues-on-your-scenery/).
When using the scenery_optimisation script, there are chances that the resulting tiles are slightly decaled in the x and/or y axis.
This decal can easily be fixed by going into the MSFS SDK, open the project, click on the "Save scenery..." button to order the scenery objects by type, then select all the tiles (not the lights, rectangles or polygons, just the tiles) and move them to the appropriate location. But this move can break some tiles altitude.
To fix it, save your scenery after moving the tiles to their appropriate location, then close your project and MSFS 2020, and run the fix_tile_altitudes script. Once the script has finished running and rebuidling your project, reopen the project in the MSFS SDK. The tiles position and altitude should now be correct.
4.2) Installation:
Just put the fix_tile_altitudes.py script, and the retrievepos.js script in a folder of your choice.
4.3) Configuration:
Change the following settings, according to your project:
- projects_folder: the parent folder that contains your sceneries
- project_name: the name of your project
- node_js_folder: the folder that contains the node js script that retrieves the Google Earth coords
- fspackagetool_folder: the folder that contains the fspackagetool exe that builds the MSFS packages
- project_file_name: the name of the xml file that embeds the project definition (by default, project_name.xml or author_name+project_name.xml)
- scene_file_name: the name of the xml file that embeds the tile descriptions (by default, objects.xml)
- package_definitions_file_name: the name of the xml file that embeds the package definitions (by default, project_name.xml or author_name+project_name.xml)
- author_name: the name of the author of the scenery
- build_package_enabled: enable the package compilation when the script has finished (default is True)
- fix_with_googleEarthDecoder_data: if set to True, tells the script to use the backup of the old objects.xml data (the one produced by the Google Earth Decoder tool) to fix tiles altitude. If set to False, tells the script to directly retrieve tiles altitude from the Google Earth API (default is True)
4.4) Usage:
Open Blender in administrator mode.
Go in the Scripting view, then click on the Open icon, and choose the fix_tile_altitudes python script.
When the configuration is done, open the Blender system console Window (Window => Toggle System Console). Then, run the script.
The fix comes with two methods: the default one uses the backup of the old objects.xml data (the one produced by the Google Earth Decoder tool)the other method tries to retrieve the altitude directly based on the Google Earth data (using the Google Earth API)
4.5) Process:
- rename the modelLib folder, in order to fix CTD issues (see https://flightsim.to/blog/creators-guide-fix-ctd-issues-on-your-scenery/)
- if necessary, install the node Js xhr2 module (npm install xhr2)
- retrieve objects positions from Google earth, via the Google Earth API, and put the results in .pos files corresponding to the scenery tiles
- update the tiles altitude, using the backup objects.xml file, if fix_with_googleEarthDecoder_data is set to True, or using the .pos files generated by the Google Earth API, if fix_with_googleEarthDecoder_data is set to False
- automatically rebuild the scenery package, if the MSFS SDK is correctly installed, and MSFS 2020 is not running
5) update_objects_LODs script:
5.1) Presentation:
This script applies the CTD issues fix (see https://flightsim.to/blog/creators-guide-fix-ctd-issues-on-your-scenery/).
This script automates the process of changing the LOD levels for all the tiles of a scenery.
5.2) Installation:
Just put the update_object_LODs.py script in a folder of your choice.
5.3) Configuration:
Change the following settings, according to your project:
- projects_folder: the parent folder that contains your sceneries
- project_name: the name of your project
- target_lods: an array representing the minsize values per LOD, starting from a minLod of 17 (from the less detailed lod to the most detailed)
- fspackagetool_folder: the folder that contains the fspackagetool exe that builds the MSFS packages
- project_file_name: the name of the xml file that embeds the project definition (by default, project_name.xml or author_name+project_name.xml)
- scene_file_name: the name of the xml file that embeds the tile descriptions (by default, objects.xml)
- package_definitions_file_name: the name of the xml file that embeds the package definitions (by default, project_name.xml or author_name+project_name.xml)
- author_name: the name of the author of the scenery
- build_package_enabled: enable the package compilation when the script has finished (default is True)
5.4) Usage:
Open Blender in administrator mode.
Go in the Scripting view, then click on the Open icon, and choose the update_object_LODs python script.
When the configuration is done, open the Blender system console Window (Window => Toggle System Console). Then, run the script.
5.5) Process:
- rename the modelLib folder, in order to fix CTD issues (see https://flightsim.to/blog/creators-guide-fix-ctd-issues-on-your-scenery/)
- update the LOD levels for each tile in the tile_object_name.xml files
- automatically rebuild the scenery package, if the MSFS SDK is correctly installed, and MSFS 2020 is not running
6) merge_sceneries script:
6.1) Presentation:
This script applies the CTD issues fix (see https://flightsim.to/blog/creators-guide-fix-ctd-issues-on-your-scenery/).
This script automates the process of merging a Google Earth Decoder source scenery into another Google Earth Decoder scenery.
6.2) Installation:
Just put the merge_sceneries.py script in a folder of your choice.
6.3) Configuration:
Change the following settings, according to your project:
- projects_folder: the parent folder that contains your sceneries
- src_project_name: the name of the scenery that you want to include in the final scenery
- dest_project_name: the name of the final project that should include both sceneries
- fspackagetool_folder: the folder that contains the fspackagetool exe that builds the MSFS packages
- src_project_file_name: the name of the xml file that embeds the source project definition (by default, src_project_name.xml or author_name+src_project_name.xml)
- dest_project_file_name: the name of the xml file that embeds the destination project definition (by default, dest_project_name.xml or author_name+dest_project_name.xml)
- src_scene_file_name: the name of the xml file that embeds the tile descriptions (by default, objects.xml) for the scenery that you want to include in the final scenery
- dest_scene_file_name: the name of the xml file that embeds the tile descriptions (by default, objects.xml) for the final project that should include both sceneries
- src_package_definitions_file_name: the name of the xml file that embeds the source package definitions (by default, src_project_name.xml or author_name+src_project_name.xml)
- dest_package_definitions_file_name: the name of the xml file that embeds the destination package definitions (by default, dest_project_name.xml or author_name+dest_project_name.xml)
- author_name: the name of the author of the scenery
- build_package_enabled: enable the package compilation when the script has finished (default is True)
6.4) Usage:
Open Blender in administrator mode.
Go in the Scripting view, then click on the Open icon, and choose the update_object_LODs python script.
When the configuration is done, open the Blender system console Window (Window => Toggle System Console). Then, run the script.
6.5) Process:
- rename the modelLib folder, in order to fix CTD issues (see https://flightsim.to/blog/creators-guide-fix-ctd-issues-on-your-scenery/)
- backup the modelLib files of the final project that should include both sceneries, into a backup subfolder, so called merge_sceneries
- copy all the xml files, gltf files, bin files and texture files from the source scenery to the destination scenery, overwritting the existing ones
- update the destination scenery scene xml file (objects.xml by default) to change the tiles guid corresponding to the source scenery tiles
- add the guid for the source tiles that does not exist in the destination scenery
- automatically rebuild the scenery package, if the MSFS SDK is correctly installed, and MSFS 2020 is not running
7) clean_package_files script:
7.1) Presentation:
This script applies the CTD issues fix (see https://flightsim.to/blog/creators-guide-fix-ctd-issues-on-your-scenery/).
This script automatically removes unused package files (.gltf, .bin and texture files), aka the files that are linked with tiles that have been removed from the scene. It appears that, if this cleaning is not done, the resulting package keep those files.
7.2) Installation:
Just put the clean_package_files.py script in a folder of your choice.
7.3) Configuration:
Change the following settings, according to your project:
- projects_folder: the parent folder that contains your sceneries
- project_name: the name of your project
- fspackagetool_folder: the folder that contains the fspackagetool exe that builds the MSFS packages
- project_file_name: the name of the xml file that embeds the project definition (by default, project_name.xml or author_name+project_name.xml)
- scene_file_name: the name of the xml file that embeds the tile descriptions (by default, objects.xml)
- package_definitions_file_name: the name of the xml file that embeds the package definitions (by default, project_name.xml or author_name+project_name.xml)
- author_name: the name of the author of the scenery
- build_package_enabled: enable the package compilation when the script has finished (default is True)
7.4) Usage:
Open Blender in administrator mode.
Go in the Scripting view, then click on the Open icon, and choose the clean_package_files python script.
When the configuration is done, open the Blender system console Window (Window => Toggle System Console). Then, run the script.
7.5) Process:
- rename the modelLib folder, in order to fix CTD issues (see https://flightsim.to/blog/creators-guide-fix-ctd-issues-on-your-scenery/)
- browse the scenery object xml files in the object folder
- check the presence of the corresponding guid in the scene xml file
- if the guid is not found in the scene xml file, remove the files associated with this scenery object xml file (.gltf, .bin, and texture files), in order to reduce the resulting package size
- if files are associated to a LOD that is not used in the .xml files, those files are now removed
8) compress_built_package script:
8.1) Prerequisites:
- This script uses the Compressonator tool from GPUOpen. You can download it here: https://github.com/GPUOpen-Tools/compressonator/releases/download/V4.1.5083/CompressonatorCLI_x64_4.1.5083.exe
- it only works on already built package. So be sure to have built your package correctly first. The package should be in the folder <path_to_projects\project_name\Packages> folder
8.2) Presentation:
This script automatically compress DDS texture files from an already built package by applying the DXT1 compression (special thks to Archer374 for suggesting me this possible optimization). It can remove transparency component from the textures, but since Google Earth tiles do not use it, as far as i know, it is lossless.
8.3) Installation:
First, install the compressonator tool in the folder of your choice. Then, put the clean_package_files.py script in a folder of your choice.
8.4) Configuration:
Change the following settings, according to your project:
projects_folder: the parent folder that contains your sceneries
project_name: the name of your project
compressonatortool_folder: the folder that contains the compressonator exe (compressonatorcli.exe) that convert and compress the DDS texture files of the packages. This exe shoud be in the <path_to_compressonator\bin\cli> folder
author_name: the name of the author of the scenery
NB_PARALLEL_TASKS: number of parallel instance of compressonator running at the same time. If you experiment issues running the script, try lowering this setting
8.5) Usage:
Open Blender in administrator mode.
Go in the Scripting view, then click on the Open icon, and choose the clean_package_files python script.
When the configuration is done, open the Blender system console Window (Window => Toggle System Console). Then, run the script.
8.6) Process:
- browse the built package for DDS files in the folder
- convert all DDS files to temporary BMP files
- convert and compress all BMP files to DXT1 format DDS files, in order to reduce the resulting package size
- remove temporary BMP files
If you want to donate, you will always be welcome to help me continue with more projects and update the existing ones https://paypal.me/Thalixte.
| 
kvari
Is there a way to directly connect to Google live instead of this ?
Thalixte 8 day(s) ago
Hello! Can you explain more precisely what you mean ?
suomy
Do you know why this happens ? I've already used (scenery optimization/retrieverspos ) and also (scenery_optimisation/decoder) neither of them recovers the positions. Thanks
https://pasteboard.co/IoqZivhlRqcC.png
Thalixte 9 day(s) ago
Yes. It could be because you have been blacklisted by Google due to too many requests to the api. What are the longitude and latitude in the xml scene file ?
suomy 9 day(s) ago
Thanks for the answer.
oh. Ok, I thought while downloading the tiles it doesn't lock the positions.
In fact at the moment I'm blocked by google.
The positions are these.
lat="41.149258131456811" lon="-8.7071679949538634 City of Porto
theluckyboy 8 day(s) ago
Bonjour les amis,
I think I have the same problem with Google.
I have this kind of error message:
"ERROR ! Google has blocked the request https://kh.google.com/rt/earth/BulkMetadata/pb=!1m2!1s!2u874"
Thalixte 8 day(s) ago
No problem on my side today. try to change your public ip address to fix this issue.
suomy 8 day(s) ago
Yesterday I bought a vpn service (ExpressVPN), but it doesn't work, everything came in blocks as I showed in the photo.
OjO 6 day(s) ago
Hi, the ban from Google should end after no more than 24h, according to my experience. How big is the area that you are trying to import? Have you tried restarting the project from scratch, and I mean by using a new folder, then GED and then Thalixte's scripts?
theluckyboy 6 day(s) ago
All is right 24 hours after the Google ban.
I dont know what is the amount of data we can download with the Google Earth Decoder to avoid the ban during the optimization or the altitude fixing.
suomy 5 day(s) ago
I know all this, and I've never had any problems doing photogrammetry, I have several scenarios here in flightsim, since I updated (SDK 1.14.1) I'm having problems, I've tried everything with the help of Thalixte and still haven't had luck and i've also redid the projects and tried in new areas and i always have this problem of getting piled up and out of coordinates.
CyberDude
@Thalixte, thanks for creating and sharing this wonderful tool!
Brandon
Hello, thanks for the amazing tool. However, is that anyway I can exclude the airport from the google photogrammetry ?
Thalixte 19 day(s) ago
Yes, once youd downloaded and optimize the tiles, go into the dev mode of MSFS, select the tiles corresponding to the airport, and remove them. Then, use terraforming polygons to blend the airport with the rest of the photogrammetry.
OjO
Hello, I made a video tutorial in English that shows how to use the scenery_optimisation script. I hope it's useful!
You find it here: https://youtu.be/6fgTxEh_x84
Thalixte 19 day(s) ago
Wonderful tutorial, very didactic and exhaustive. Just a remark: you can download more tiles from the msfs decoder tools. In my rig, i have 32Gb of RAM, and i can use a rectangle that covers approximatively 15000 data without any problems, with 17-19 Lods. More data can be problematic, but until 15000, it is good.
Also, Thanks for the "snap to ground" trick to correct tile elevation issues when moving them. Never thought about that. I made a specific script for that purpose (fix_tiles_altitude.py), but your tricks seems to make it easier to apply the correction 😉 .
Also, great colorimetry settings for the Benevento scenery. I guess i will try them for some future creations.
I added your video in the description of the tools.
OjO 19 day(s) ago
Thank you for the scripts. They are actually easy to use, once you know how to deal with scripts in Blender. And the results are great.
The color correction values are actually taken from the Brazilian video I used as a base for my tutorial (I think I said so in the video in Italian, but didn't do so in the English one - I'll put it in the description), so all the merits go to Rodukao.
The snap to ground technique is all mine, though 😋. I experimented a bit because I couldn't use your altitude script in the right way and I'm glad I could come up with a solution that can be applied while editing the scenery in MSFS. I've no idea why it works, but it does, so... no questions asked!
Regarding the number of tiles. I've kept the number extremely low because the problem in my case wasn't the RAM (I've got 32GB), but the part when the script tries to retrieve the coordinates from google. I experimented with 8000 tiles and every time I tried I had my IP blocked and/or all the tiles ended up stacked at the same coordinates (I've a fast internet connection, so it shouldn't be related to a poor connection). I've tried with the [0,0,0] pos file but couldn't solve the issue, so I decided to stick with smaller numbers and rely on the merge script. As I say in the video, people can try higher sizes, but I wanted to suggest those that gave me the smallest amount of issues (or none at all).
Speaking of the merge script... I was planning to do a short tutorial about it too, but I've got some strange issues with it, especially with altitudes (but there's also a problem with some smaller tiles disappearing too - it seems to me that it's a problem with the LOD, but I'm not sure). I want to test it a little bit more, but eventually I might have to ask you. Should I send you a PM, in case?
Thalixte 19 day(s) ago
Yes you can, for sure. As for th number of data, it is different from the number of tiles. The number of tiles correspond more to the number of metadata (the first retrieving step of the MSFS decoder). For the number of tiles, of course, it is better not to exceed 1500 tiles.
CyberDude 14 day(s) ago
@Signor Ojo, thank you very much for the EXCELLENT tutorial!
You have explained everything so well!I
OjO 13 day(s) ago
@CyberDude Thanks for the feedback!
theluckyboy
Bonjour Thalixte,
Merci pour ces scripts que tu as mis en place.
J'ai réussi à bien suivre tes instructions, mais j'ai un problème de generation de bâtiments sur la scène que je souhaite créer.
Comment empêcher l'autogeneration de bâtiments?
par exmple: https://ibb.co/pWBH8MB
Merci d'avance
Thalixte 24 day(s) ago
Bonjour. As-tu créé un polygone d'exclusion pour que le bâtiment auto généré n'apparaisse pas ?
theluckyboy 24 day(s) ago
Non, merci pour la réponse
theluckyboy 3 day(s) ago
Bonjour,
J'ai finalement réussi à créer quelques scènes de Blagnac et de ses environs (Purpan, En-Jacca) et aussi Labège. Maintenant, reste à améliorer la gestion des couleurs ...
Voici une petite vidéo du résultat: https://youtu.be/u5-TX0hcl9I
Excellent weekend.
Perschistence
What happens if you have two projects which overlap themselves in terms of the area they cover and merge them with the ,,merge_sceneries.py" script? Is it going to retain all the tiles, or will it delete them from one project?
Thalixte 2 month(s) ago
Here is an illustration of the quickly made Frankfurt scenery:
https://live.staticflickr.com/65535/51346108664_8a0d44ade5_3k.jpg
https://live.staticflickr.com/65535/51345362016_98d6b54d5f_3k.jpg
The link to get it: https://drive.google.com/file/d/1TQ8z8oGMw0Zl6BYzp7zxWHZQm65DZnKu/view?usp=sharing
The merge_sceneries keep the tiles from the first scenery indicated in the settings, and remove the corresponding tiles from the second one. To sum up, the second scenery indicated in the settings should always be the target scenery (the one that will be used to produce the package).
Some infos to compare (remember my scenery covers a bigger area):
number of files in the package:
size of the packages:
Perschistence 2 month(s) ago
The screenshots look good, I've got to say! Nice to know, because then it would not make any problems to import smaller parts of a big city and combine the projects later, in the end!
Could I for example import a city in one project with LOD 19 and in another project the center of the city with LOD 21, combine them in the end and get just the center of the city in full detail?
Thalixte 2 month(s) ago
Yes, of course, that's what i do regurarly. But i cannot make 17-21 tiles (too big when building and optimizing). I combine mostly 17-19 tiles with some 17-20 tiles, and it is already great. For instance, in this Frankfurt scenery, i retrieved some 17-20 tiles for the downtown, and the historical center. Later, i surely will merge them with the scenery i already brought to you (which is only 17-19 for the moment)...
Perschistence 2 month(s) ago
Uhh, nice, could you send me the final one too? Would be cool for screenshots^^
Thalixte 2 month(s) ago
There is no final one for the moment. And i don't know if i will release it soon, because there will be the next World Update on Germany, and maybe MS/Asobo will bring photogrammetry for this city, i don't know. But you can already enjoy the one i brought to you in the link.
Perschistence 2 month(s) ago
Thank you! 😊
Perschistence 2 month(s) ago
Do I get this right? When I have imported a town with LOD 19 and the center of the town in another project with LOD 21, I need to use the LOD19 as the dest_project_name and the LOD21 as src_project_name? Because I want to merge the LOD21 into the ,,destination" LOD19.
Should I first merge them and then run the optimization script on the final package, or should I first optimize both projects and then merge them together?
Thalixte 2 month(s) ago
First, optimize them, then merge them, and yes, you are right on the settings. Unsure also that both sceneries have the same min lod (17 for instance)
Perschistence 2 month(s) ago
Sorry, I don't understand your last sentence^^ Do all the projects need to have the same (lowest) LOD?
Thalixte 2 month(s) ago
Yes, they should
Perschistence 2 month(s) ago
I am encountering a problem at the moment. I try to merge a LOD 16 - 20 project with an LOD 16 - 21 Project but after about 6 minutes the FlightSimulator.exe console window is just closing itself without any error message, and it does not complete building the package. If I try to build it in game, the same happens after about 6 minutes. The game just closes down without any error message.
Thalixte 2 month(s) ago
Hi! This is because tou used 16 as min-lod. Using 16 will create bigger tiles thanu using 17, so it will leads to a too big cosmption of resources on your rig, and will crash. I strongly suggest to use 17 as min lods to produce tiles with correct dmensions..I sometimes use 16 as min lods in some sceneries (Thesaloniki for instance), and it worked, but it is very limit, especially if you use 21 as max lods. I personnaly never use 21 as max lods, as it will produce too big tiles, with too much meshes and vertices. I am also finishing the Frankfurt scenery (i hope you don't work on a similar one), as it is great to have this city in photogrammetry (i only hope Asobo will not anounce it to be in photogrammetry in the newt Wolrd Update).
Perschistence 2 month(s) ago
I now did merge some LOD 16-20 Projects with some 16-21 Projects and it did work for some reason. I guess my project yesterday was too big because it is trying to compile it in memory, as it says. Maybe they've set a limit. The package sources should probably stay well below 15 GB, but I am not 100% sure. I really did work on another Frankfurt project, but I liked yours even better than my actual one, just the LOD was too low. I merged LOD 16-21 for the center of the city with LOD 16-20 on the outside. I tried to combine it with LOD 16- 19 for the outside of the border of the city, but it is not compiling anymore. Maybe the project was too big again. Here you can see my try on a HQ version of Frankfurt but I would love to see yours too. Unfortunately, with my ,,brute force approach", I had to make mine a bit smaller. https://www.forum-3dcenter.org/vbulletin/showpost.php?p=12753846&postcount=2305
Thalixte 2 month(s) ago
Do you mind if i publish my scenery this week ? In my last version, the downtown is more detailed (min LOD of 20)
Perschistence 2 month(s) ago
Cool, why should I mind? I think more options are a good thing!
Thalixte 2 month(s) ago
Cool. So do you want me to add some POI in my scenery ? I don't know Frankfurt as well as you do, i guess. Maybe you can help me
Perschistence 2 month(s) ago
Hmm, I am not sure. You could use the highest buildings: https://www.wikiwand.com/en/List_of_tallest_buildings_in_Frankfurt
Thalixte 2 month(s) ago
Yes, that's what i did 😊
Perschistence 2 month(s) ago
Nice, I am looking forward trying your mod. 😊
Perschistence
I have an interesting question. There are some mods on flightsim.to, for example Frankfurt (Germany). They are about 1GB in size. When I try to do the same and import the whole town, my project gets about 10GB in size, or even 20GB. I can go down to LOD 16-19, then I get something that is just a few GB in size, but the quality of my town is way worse (and the project is still bigger). How can you achieve a size of just [email protected] like many city mods here and make it still look like LOD 21 or so? Maybe you can load it in blender first or something and massively simplify it?
I further investigated it.
My packages have a .bgl file what is multiple GB big in the scenery folder. The projects from flightsim.io also have a .bgl file but it is just a few kb in size. What is this?
Thalixte 2 month(s) ago
Hi! First of all, i rarely use 16 as a min LOD, because it produces bigger tiles, that are difficult to handle with Blender, and in the max level of details (21) are too big and cannot be built by the package builder.
Secondly, i mostly use 17-19 LODS, and in some specific places, i use 17-20 (to get a bette level of details near the ground). 17-21 is very very resource consuming. I suggest to use it on a landmark, not on a whole tile.
As of Frunkfurt scenery, according to the feedback, it does not seem to be optimized.
Just tested the Frankfurt scenery, and the area is not as big as i thought, so it explains the size of the package. Also, there are only 3 LODS per tiles, so i guess the author used 17-19 or 18-20 lod settings.
It is also not optimized considering the LOD spheres. In my rig, i have difficulties to maintain 30 fps with this scenery (i have an average of 27-28 fps, with the SU5 installed).
Perschistence 2 month(s) ago
Yes, the Frankfurt-Scenery performs terribly, it even tanks your framerate if you are 20 miles away.
If you take a look at the debug view you can see that it is nearly all the time using the highest LOD, you have to fly very, very far away to let it switch to the next one. The creator should have used your tools. 😉
I wish there would be a way to reconfigure the LOD for finished projects.
But if you try to import the whole town like in his 2 projects, even if you just use LOD 16-19 the project is going to be multiple times bigger. And his mod looks mostly like LOD 20. LOD 19 is way more ,,molten" and ,,blurred".
60% of the size is made up of the [Project_Name]-modelLib.BGL file, what is just about a few KB in a lot of downloadable mods like the frankfurt projects. Do you know what this is or have an idea if we could get rid of it?
It would save 60% space, what would be enormous.
Edit: Oh, now I see that he also has a pretty big BGL file. Maybe he just deleted way more stuff he does not need...
Thalixte 2 month(s) ago
I am making a quickly done scenery of Frankfurt. It is already optimized now, and covers a little bigger area than the addon you pointed me. When finished, i will give you a link so you can test and see the difference with an optimized scenery.
Perschistence 2 month(s) ago
Wow, that would be great, thank you very much!
Thalixte 2 month(s) ago
Ok, so here it is: https://drive.google.com/file/d/1TQ8z8oGMw0Zl6BYzp7zxWHZQm65DZnKu/view?usp=sharing
Perschistence 2 month(s) ago
Thank you! I am going to try it out as soon as I can. Wow, you brought it down to 626MB. I ask myself, is the 7z format so much more efficient than zip or rar?
Thalixte 2 month(s) ago
Yes, it can be
Perschistence
Cannot execute the compress_built_package.py because when I try it, this happens: https://i.ibb.co/Jq85XnM/Another-Instance.png
When I then lower NB_PARALLEL_TASKS to just 1 it also does not work and always shows "invalid commandline".
Thalixte 2 month(s) ago
You must install compressonator for this script to work. This script only compress DDS textures on an already built package. As mentioned in the section 8 of the description (or in the README.md file in the github repo), the script do the following:
Process:
Perschistence 2 month(s) ago
Do I have to install the compressonator in any specific way? I downloaded the .exe file from your link: https://github.com/GPUOpen-Tools/compressonator/releases/download/V4.1.5083/CompressonatorCLI_x64_4.1.5083.exe renamed it to "compressonatorcli.exe" placed it at "F:\\Google-Earth-Decoder-optimization-tools" and configured the "compress_built_package.py" as following:
After executing, I encountered the problem from my screenshot.
Perschistence 2 month(s) ago
Ahh, I now see, the Compressonator-File is a Setup^^
Perschistence
Sorry for writing so many comments. The JPEG compression seems to have no impact on the final package, because in the end it is converted to a DDS file what has exactly the same size. Or is it not meant to decrease the size of the final package?
Thalixte 2 month(s) ago
The JPG compression can have an impact if you change the compression setting. But it is not a losless process. That's why i prefer to keep PNG texture files
Perschistence 2 month(s) ago
I've set the "output_texture_format" to "JPG_FORMAT" and did not change the "JPG_COMPRESSION_RATIO" to something other than 0.75. Shouldn't this have changed the file sizes?
Thalixte 2 month(s) ago
I suggest you to rollback the texture_format to PNG_FORMAT.
Perschistence 2 month(s) ago
Hmm I am now running the optimization of another project and sometimes I see this in the log:
It is running for hours now, I ask myself if I can simply ignore it or if I have to restart the whole process again? Most of the time it seems to work. At least I get a lot of JPEG files. It wouldn't be too bad if I would just be missing some textures I guess...
I would simply stay with PNG compression but my project is 10GB in size and even the package compression in the end just reduces this by 10%. I guess, there is alot of potential to save storage and maybe even RAM with JPEG. Hmm, maybe I could do the compression manually via a batch script or something if it does not work.
Thalixte 2 month(s) ago
I think there is a miseunderstood: the optimization process is not just the fact of compress jpeg files.
First of all, 10 Gb is too heavy, even for my optimization scripts. I mostly use sceneries with less than 6 Gob of data. To do so, i split my sceneries in different packages.
The, the texture optimisation uses the Lily texture Packer in order to get only one texture per tile and LOD level. If you look at your package, you will see that, for one tile and one LOD level, you have a bunch of texture files.
The optimization process will reduce the number of files by associate only one texture file with one tile and LOD level. This feature help reducing the loading time of a scenery (until SU5, but i did not test it for the moment, currently downloading the 40 Gb update).
The optimization process will also redimension the tile bpunding box in order to be efficient with the LOD managament system of the simulation. This feature helps keeping stable framerate, even with an heavy photogrammetric scenery.
The optimization process also ensure that all tiles willl be correctly built, for instance, by converting all the textures in the same format (PNG or JPG, as you want, but i advice to use PNG, because in the end, if tou use the compress_built_package script, you will get optimized texture file size).
And, at last, the optimization process will update the gltf files to make them compatible with road traffic, and collision detection.
Perschistence 2 month(s) ago
I guess it is possible to save more space with JPEG than with the scenery_optimisation.py script if I accept a degradation in quality and therefore I shouldn't use the scenery optimisation script. But I don't know what the implications for the game are because JPEG first has to get decoded and is not meant for real-time game stuff. On the other hand, it could even perform better if ram or disc bandwidth is a problem and CPUs are pretty fast in decoding JPEGs.
Thalixte 2 month(s) ago
Guess what ? With the last update, memory footprint seems lighter. So, it is maybe not necessary to lose texture quality like this
Perschistence 2 month(s) ago
Seems amazing, +35% performance in 4k and +82% in 1080p. -66% ram usage. At least that is what golem.de measured: https://www.golem.de/news/flight-simulator-im-benchmark-test-sim-update-5-laesst-performance-abheben-2107-158478.html I am going to see if it also applies to most of the mods.
Perschistence
Hmm, I enabled bake_textures_enabled now. But my output package has exactly as many files as the one where it was disabled?? Even the same filesize.
Thalixte 2 month(s) ago
Normally, it should not. if you selected a 17-20 LOD scenery, the number of texture should be reduced by a factor of ten. Have you opened the windows console in Blender ? When you run the script, all the process is detailed in the logs
Perschistence 2 month(s) ago
Yes, I always have opened the logs, even though I am doing something different on the side. Should I look for something special in die output logs the next time? At least I saw the names were changed in the resulting package, PNG was replaced with JPG, but there were exactly as many files as before with even the same size down to the KB.
Thalixte 2 month(s) ago
Normally, the tiles must be optimized by the process. You should have in your log a section called "OPTIMIZE SCENERY OBJECTS"
Perschistence
I guess if I already executed the "scenery_optimisation.py" script I don't have to run the "update_object_LODs.py" anymore?
Thalixte 2 month(s) ago
Yes, it is not necessary, except if you plan to change the LOD settings of the scenery
Perschistence
I imported my hometown with LOD 17 - 20 and used the scenery optimization and update object LOD script. (I just could not use the bake_textures_enabled until now.) When I start the game, I have around 30FPS but if I fly over the imported town for a while the framerate drops to 7 - 15FPS and even if I fly 20Km away from the town, performance stays low. The town seems to get never unloaded? Is there anything I can do? I wanted to import way more stuff. I have an RTX 3090, 64GB Ram and an i9 9900k. The game uses more than 20GB of Vram, if I fly long enough and the GPU usage is just about 50%, what indicates a CPU bottleneck.
Thalixte 2 month(s) ago
You need to optimize your scenery first. Changing the LOD will only be useful if you optimized the tiles with my optimization script.
Perschistence
I ran the script and I've got the following error:
I have LiliPackedImage installed, how could I fix this?
Thalixte 2 month(s) ago
Hi! This seems that the script does not find the Lilytexturepacker addon. Have you checked it is correctly installed ? In the Object Menu, you must have two new tries: Lily Texture Packer and Lily Capture Merger. Also, have you checked the version of Blender. This should be 2.83
Perschistence 2 month(s) ago
Thank you very much for your reply! My Blender Version is 2.83.9. The Lili Texture Packer seems to be installed, take a look at this screenshot: https://i.ibb.co/GFkP6DP/Blender-Add-Ons.png Is the version maybe too new? It works if I set "bake_textures_enabled = False", but it is a very nice feature, I would love to use it. 😊 Here is the menu entry: https://i.ibb.co/87Gh22q/Lily-Texture-Packer.png
Thalixte 2 month(s) ago
I wonder if it is not a matter of Lily Texture Packer version. Mine is 1.1.1. It seems that the addon code has changed to be compatible with Blender v2.93
Perschistence 2 month(s) ago
I am going to try it with an older version later. 😊
Perschistence 2 month(s) ago
It works with version 1.1.x, thanks for improving the readme!
captainShokomon 2 month(s) ago
No matter which version of Lily Texture Packer I install, I do not see an entry in the Blender objects menu that says Lily Capture Merger, only Lily Texture packer. There is a separate Gumroad page to buy Lily Capture Merger - did you guys have to pay for it separately? Or did it appear in the menu when you installed Lily Texture Packer>
Thalixte 2 month(s) ago
Yes, you're right. That's just another addon that i retrieved. Don't care about it
SakuraSayaka
So, now I would only search for "Thalixte" and download the large-scale scenery optimized by this tool. Thanks for the great work😄
Thalixte 4 month(s) ago
JBOY76
I am getting blocked all the time even if a change ip number with my VPN !
Regards Johnny B.
Thalixte 4 month(s) ago
JBOY76 4 month(s) ago
jeanmi1951
Thalixte 4 month(s) ago
Il s'agit de toutes les textures de la scène.
jeanmi1951
Tout d' abord mon soutien par rapport à toute ces critiques,chose la plus facile à faire...
Je me lance avec Google et j'aimerais savoir comment corriger la colorimétrie,et avec quel logiciel.
Merci à l'avance
JM
Thalixte 5 month(s) ago
Lash123456
I tried out two scripts for the first time and have problems
Compress_built_package script
I get an error in the console saying my textures folder is not found..... though I have completed the fields correctly
Merge scenery script
Kind of works, but the finished scenery hasnt brought over the exclusions and terraforming from the folder I just merged
Thalixte 6 month(s) ago
For the compress_built_packages, the script assume that the textures are on the \scenery\global\scenery\TEXTURE of the built package.
Lash123456
https://ibb.co/q1JLwPd
Thalixte 6 month(s) ago
Lash123456 6 month(s) ago
Thalixte 6 month(s) ago