Improved Configuration and YAML Management (#2966)

# Description of Changes

**What was changed:**
- **Configuration Updates:**  
Replaced all calls to `GeneralUtils.saveKeyToConfig` with the new
`GeneralUtils.saveKeyToSettings` method across multiple classes (e.g.,
`LicenseKeyChecker`, `InitialSetup`, `SettingsController`, etc.). This
update ensures consistent management of configuration settings.
  
- **File Path and Exception Handling:**  
Updated file path handling in `SPDFApplication` by creating `Path`
objects from string paths and logging these paths for clarity. Also
refined exception handling by catching more specific exceptions (e.g.,
using `IOException` instead of a generic `Exception`).

- **Analytics Flag and Rate Limiting:**  
Changed the analytics flag in the application properties from a `String`
to a `Boolean`, and updated related logic in `AppConfig` and
`PostHogService`. The rate-limiting property retrieval in `AppConfig`
was also refined for clarity.

- **YAML Configuration Management:**  
Replaced the previous manual, line-based YAML merging logic in
`ConfigInitializer` with a new `YamlHelper` class. This helper leverages
the SnakeYAML engine to load, update, and save YAML configurations more
robustly while preserving comments and formatting.

**Why the change was made:**
- **Improved Maintainability:**  
Consolidating configuration update logic into a single utility method
(`saveKeyToSettings`) reduces code duplication and simplifies future
maintenance.
  
- **Enhanced Robustness:**  
The new `YamlHelper` class ensures that configuration files are merged
accurately and safely, minimizing risks of data loss or format
corruption.
  
- **Better Type Safety and Exception Handling:**  
Switching the analytics flag to a Boolean and refining exception
handling improves code robustness and debugging efficiency.
  
- **Clarity and Consistency:**  
Standardizing file path handling and logging practices enhances code
readability across the project.

**Challenges encountered:**
- **YAML Merging Complexity:**  
Integrating the new `YamlHelper` required careful handling to preserve
existing settings, comments, and formatting during merges.
  
- **Type Conversion and Backward Compatibility:**  
Updating the analytics flag from a string to a Boolean required
extensive testing to ensure backward compatibility and proper
functionality.
  
- **Exception Granularity:**  
Refactoring exception handling from a generic to a more specific
approach involved a detailed review to cover all edge cases.

Closes #<issue_number>

---

## Checklist

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#6-testing)
for more details.

---------

Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
This commit is contained in:
Ludy
2025-02-25 22:52:59 +01:00
committed by GitHub
parent 4fabc07a44
commit ac10c9fa43
12 changed files with 540 additions and 391 deletions

View File

@@ -51,7 +51,7 @@ public class LicenseKeyChecker {
public void updateLicenseKey(String newKey) throws IOException {
applicationProperties.getEnterpriseEdition().setKey(newKey);
GeneralUtils.saveKeyToConfig("EnterpriseEdition.key", newKey, false);
GeneralUtils.saveKeyToSettings("EnterpriseEdition.key", newKey);
checkLicense();
}

View File

@@ -83,17 +83,18 @@ public class SPDFApplication {
Map<String, String> propertyFiles = new HashMap<>();
// External config files
String settingsPath = InstallationPathConfig.getSettingsPath();
log.info("Settings file: {}", settingsPath);
if (Files.exists(Paths.get(settingsPath))) {
propertyFiles.put("spring.config.additional-location", "file:" + settingsPath);
Path settingsPath = Paths.get(InstallationPathConfig.getSettingsPath());
log.info("Settings file: {}", settingsPath.toString());
if (Files.exists(settingsPath)) {
propertyFiles.put(
"spring.config.additional-location", "file:" + settingsPath.toString());
} else {
log.warn("External configuration file '{}' does not exist.", settingsPath);
log.warn("External configuration file '{}' does not exist.", settingsPath.toString());
}
String customSettingsPath = InstallationPathConfig.getCustomSettingsPath();
log.info("Custom settings file: {}", customSettingsPath);
if (Files.exists(Paths.get(customSettingsPath))) {
Path customSettingsPath = Paths.get(InstallationPathConfig.getCustomSettingsPath());
log.info("Custom settings file: {}", customSettingsPath.toString());
if (Files.exists(customSettingsPath)) {
String existingLocation =
propertyFiles.getOrDefault("spring.config.additional-location", "");
if (!existingLocation.isEmpty()) {
@@ -101,9 +102,11 @@ public class SPDFApplication {
}
propertyFiles.put(
"spring.config.additional-location",
existingLocation + "file:" + customSettingsPath);
existingLocation + "file:" + customSettingsPath.toString());
} else {
log.warn("Custom configuration file '{}' does not exist.", customSettingsPath);
log.warn(
"Custom configuration file '{}' does not exist.",
customSettingsPath.toString());
}
Properties finalProps = new Properties();
@@ -154,7 +157,7 @@ public class SPDFApplication {
} else if (os.contains("nix") || os.contains("nux")) {
SystemCommand.runCommand(rt, "xdg-open " + url);
}
} catch (Exception e) {
} catch (IOException e) {
log.error("Error opening browser: {}", e.getMessage());
}
}

View File

@@ -96,9 +96,9 @@ public class AppConfig {
@Bean(name = "rateLimit")
public boolean rateLimit() {
String appName = System.getProperty("rateLimit");
if (appName == null) appName = System.getenv("rateLimit");
return (appName != null) ? Boolean.valueOf(appName) : false;
String rateLimit = System.getProperty("rateLimit");
if (rateLimit == null) rateLimit = System.getenv("rateLimit");
return (rateLimit != null) ? Boolean.valueOf(rateLimit) : false;
}
@Bean(name = "RunningInDocker")
@@ -170,16 +170,14 @@ public class AppConfig {
@Bean(name = "analyticsPrompt")
@Scope("request")
public boolean analyticsPrompt() {
return applicationProperties.getSystem().getEnableAnalytics() == null
|| "undefined".equals(applicationProperties.getSystem().getEnableAnalytics());
return applicationProperties.getSystem().getEnableAnalytics() == null;
}
@Bean(name = "analyticsEnabled")
@Scope("request")
public boolean analyticsEnabled() {
if (applicationProperties.getEnterpriseEdition().isEnabled()) return true;
return applicationProperties.getSystem().getEnableAnalytics() != null
&& Boolean.parseBoolean(applicationProperties.getSystem().getEnableAnalytics());
return applicationProperties.getSystem().isAnalyticsEnabled();
}
@Bean(name = "StirlingPDFLabel")

View File

@@ -9,7 +9,6 @@ import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
import java.util.*;
import lombok.extern.slf4j.Slf4j;
@@ -37,7 +36,6 @@ public class ConfigInitializer {
log.info("Created settings file from template");
} else {
// 2) Merge existing file with the template
Path settingsPath = Paths.get(InstallationPathConfig.getSettingsPath());
URL templateResource = getClass().getClassLoader().getResource("settings.yml.template");
if (templateResource == null) {
throw new IOException("Resource not found: settings.yml.template");
@@ -49,160 +47,33 @@ public class ConfigInitializer {
Files.copy(in, tempTemplatePath, StandardCopyOption.REPLACE_EXISTING);
}
// 2a) Read lines from both files
List<String> templateLines = Files.readAllLines(tempTemplatePath);
List<String> mainLines = Files.readAllLines(settingsPath);
// Copy setting.yaml to a temp location so we can read lines
Path settingTempPath = Files.createTempFile("settings", ".yaml");
try (InputStream in = Files.newInputStream(destPath)) {
Files.copy(in, settingTempPath, StandardCopyOption.REPLACE_EXISTING);
}
// 2b) Merge lines
List<String> mergedLines = mergeYamlLinesWithTemplate(templateLines, mainLines);
YamlHelper settingsTemplateFile = new YamlHelper(tempTemplatePath);
YamlHelper settingsFile = new YamlHelper(settingTempPath);
// 2c) Only write if there's an actual difference
if (!mergedLines.equals(mainLines)) {
Files.write(settingsPath, mergedLines);
boolean changesMade =
settingsTemplateFile.updateValuesFromYaml(settingsFile, settingsTemplateFile);
if (changesMade) {
settingsTemplateFile.save(destPath);
log.info("Settings file updated based on template changes.");
} else {
log.info("No changes detected; settings file left as-is.");
}
Files.deleteIfExists(tempTemplatePath);
Files.deleteIfExists(settingTempPath);
}
// 3) Ensure custom settings file exists
Path customSettingsPath = Paths.get(InstallationPathConfig.getCustomSettingsPath());
if (!Files.exists(customSettingsPath)) {
if (Files.notExists(customSettingsPath)) {
Files.createFile(customSettingsPath);
log.info("Created custom_settings file: {}", customSettingsPath.toString());
}
}
/**
* Merge logic that: - Reads the template lines block-by-block (where a "block" = a key and all
* the lines that belong to it), - If the main file has that key, we keep the main file's block
* (preserving whitespace + inline comments). - Otherwise, we insert the template's block. - We
* also remove keys from main that no longer exist in the template.
*
* @param templateLines lines from settings.yml.template
* @param mainLines lines from the existing settings.yml
* @return merged lines
*/
private List<String> mergeYamlLinesWithTemplate(
List<String> templateLines, List<String> mainLines) {
// 1) Parse template lines into an ordered map: path -> Block
LinkedHashMap<String, Block> templateBlocks = parseYamlBlocks(templateLines);
// 2) Parse main lines into a map: path -> Block
LinkedHashMap<String, Block> mainBlocks = parseYamlBlocks(mainLines);
// 3) Build the final list by iterating template blocks in order
List<String> merged = new ArrayList<>();
for (Map.Entry<String, Block> entry : templateBlocks.entrySet()) {
String path = entry.getKey();
Block templateBlock = entry.getValue();
if (mainBlocks.containsKey(path)) {
// If main has the same block, prefer main's lines
merged.addAll(mainBlocks.get(path).lines);
} else {
// Otherwise, add the template block
merged.addAll(templateBlock.lines);
}
}
return merged;
}
/**
* Parse a list of lines into a map of "path -> Block" where "Block" is all lines that belong to
* that key (including subsequent indented lines). Very naive approach that may not work with
* advanced YAML.
*/
private LinkedHashMap<String, Block> parseYamlBlocks(List<String> lines) {
LinkedHashMap<String, Block> blocks = new LinkedHashMap<>();
Block currentBlock = null;
String currentPath = null;
for (String line : lines) {
if (isLikelyKeyLine(line)) {
// Found a new "key: ..." line
if (currentBlock != null && currentPath != null) {
blocks.put(currentPath, currentBlock);
}
currentBlock = new Block();
currentBlock.lines.add(line);
currentPath = computePathForLine(line);
} else {
// Continuation of current block (comments, blank lines, sub-lines)
if (currentBlock == null) {
// If file starts with comments/blank lines, treat as "header block" with path
// ""
currentBlock = new Block();
currentPath = "";
}
currentBlock.lines.add(line);
}
}
if (currentBlock != null && currentPath != null) {
blocks.put(currentPath, currentBlock);
}
return blocks;
}
/**
* Checks if the line is likely "key:" or "key: value", ignoring comments/blank. Skips lines
* starting with "-" or "#".
*/
private boolean isLikelyKeyLine(String line) {
String trimmed = line.trim();
if (trimmed.isEmpty() || trimmed.startsWith("#") || trimmed.startsWith("-")) {
return false;
}
int colonIdx = trimmed.indexOf(':');
return (colonIdx > 0); // someKey:
}
// For a line like "security: ", returns "security" or "security.enableLogin"
// by looking at indentation. Very naive.
private static final Deque<String> pathStack = new ArrayDeque<>();
private static int currentIndentLevel = 0;
private String computePathForLine(String line) {
// count leading spaces
int leadingSpaces = 0;
for (char c : line.toCharArray()) {
if (c == ' ') leadingSpaces++;
else break;
}
// assume 2 spaces = 1 indent
int indentLevel = leadingSpaces / 2;
String trimmed = line.trim();
int colonIdx = trimmed.indexOf(':');
String keyName = trimmed.substring(0, colonIdx).trim();
// pop stack until we match the new indent level
while (currentIndentLevel >= indentLevel && !pathStack.isEmpty()) {
pathStack.pop();
currentIndentLevel--;
}
// push the new key
pathStack.push(keyName);
currentIndentLevel = indentLevel;
// build path by reversing the stack
String[] arr = pathStack.toArray(new String[0]);
List<String> reversed = Arrays.asList(arr);
Collections.reverse(reversed);
return String.join(".", reversed);
}
/**
* Simple holder for the lines that comprise a "block" (i.e. a key and its subsequent lines).
*/
private static class Block {
List<String> lines = new ArrayList<>();
}
}

View File

@@ -44,7 +44,7 @@ public class InitialSetup {
if (!GeneralUtils.isValidUUID(uuid)) {
// Generating a random UUID as the secret key
uuid = UUID.randomUUID().toString();
GeneralUtils.saveKeyToConfig("AutomaticallyGenerated.UUID", uuid);
GeneralUtils.saveKeyToSettings("AutomaticallyGenerated.UUID", uuid);
applicationProperties.getAutomaticallyGenerated().setUUID(uuid);
}
}
@@ -54,7 +54,7 @@ public class InitialSetup {
if (!GeneralUtils.isValidUUID(secretKey)) {
// Generating a random UUID as the secret key
secretKey = UUID.randomUUID().toString();
GeneralUtils.saveKeyToConfig("AutomaticallyGenerated.key", secretKey);
GeneralUtils.saveKeyToSettings("AutomaticallyGenerated.key", secretKey);
applicationProperties.getAutomaticallyGenerated().setKey(secretKey);
}
}
@@ -64,8 +64,8 @@ public class InitialSetup {
"0.36.0", applicationProperties.getAutomaticallyGenerated().getAppVersion())) {
Boolean csrf = applicationProperties.getSecurity().getCsrfDisabled();
if (!csrf) {
GeneralUtils.saveKeyToConfig("security.csrfDisabled", false, false);
GeneralUtils.saveKeyToConfig("system.enableAnalytics", "true", false);
GeneralUtils.saveKeyToSettings("security.csrfDisabled", false);
GeneralUtils.saveKeyToSettings("system.enableAnalytics", true);
applicationProperties.getSecurity().setCsrfDisabled(false);
}
}
@@ -76,14 +76,14 @@ public class InitialSetup {
String termsUrl = applicationProperties.getLegal().getTermsAndConditions();
if (StringUtils.isEmpty(termsUrl)) {
String defaultTermsUrl = "https://www.stirlingpdf.com/terms-and-conditions";
GeneralUtils.saveKeyToConfig("legal.termsAndConditions", defaultTermsUrl, false);
GeneralUtils.saveKeyToSettings("legal.termsAndConditions", defaultTermsUrl);
applicationProperties.getLegal().setTermsAndConditions(defaultTermsUrl);
}
// Initialize Privacy Policy
String privacyUrl = applicationProperties.getLegal().getPrivacyPolicy();
if (StringUtils.isEmpty(privacyUrl)) {
String defaultPrivacyUrl = "https://www.stirlingpdf.com/privacy-policy";
GeneralUtils.saveKeyToConfig("legal.privacyPolicy", defaultPrivacyUrl, false);
GeneralUtils.saveKeyToSettings("legal.privacyPolicy", defaultPrivacyUrl);
applicationProperties.getLegal().setPrivacyPolicy(defaultPrivacyUrl);
}
}
@@ -97,7 +97,7 @@ public class InitialSetup {
appVersion = props.getProperty("version");
} catch (Exception e) {
}
GeneralUtils.saveKeyToSettings("AutomaticallyGenerated.appVersion", appVersion);
applicationProperties.getAutomaticallyGenerated().setAppVersion(appVersion);
GeneralUtils.saveKeyToConfig("AutomaticallyGenerated.appVersion", appVersion, false);
}
}

View File

@@ -0,0 +1,479 @@
package stirling.software.SPDF.config;
import java.io.IOException;
import java.io.StringWriter;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayDeque;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Deque;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Optional;
import java.util.Set;
import java.util.function.Function;
import org.snakeyaml.engine.v2.api.Dump;
import org.snakeyaml.engine.v2.api.DumpSettings;
import org.snakeyaml.engine.v2.api.LoadSettings;
import org.snakeyaml.engine.v2.api.StreamDataWriter;
import org.snakeyaml.engine.v2.common.FlowStyle;
import org.snakeyaml.engine.v2.common.ScalarStyle;
import org.snakeyaml.engine.v2.composer.Composer;
import org.snakeyaml.engine.v2.nodes.MappingNode;
import org.snakeyaml.engine.v2.nodes.Node;
import org.snakeyaml.engine.v2.nodes.NodeTuple;
import org.snakeyaml.engine.v2.nodes.ScalarNode;
import org.snakeyaml.engine.v2.nodes.SequenceNode;
import org.snakeyaml.engine.v2.nodes.Tag;
import org.snakeyaml.engine.v2.parser.ParserImpl;
import org.snakeyaml.engine.v2.scanner.StreamReader;
import lombok.extern.slf4j.Slf4j;
@Slf4j
public class YamlHelper {
// YAML dump settings with comment support and block flow style
private static final DumpSettings DUMP_SETTINGS =
DumpSettings.builder()
.setDumpComments(true)
.setWidth(Integer.MAX_VALUE)
.setDefaultFlowStyle(FlowStyle.BLOCK)
.build();
private final String yamlContent; // Stores the entire YAML content as a string
private LoadSettings loadSettings =
LoadSettings.builder()
.setUseMarks(true)
.setMaxAliasesForCollections(Integer.MAX_VALUE)
.setAllowRecursiveKeys(true)
.setParseComments(true)
.build();
private Path originalFilePath;
private Node updatedRootNode;
// Constructor with custom LoadSettings and YAML string
public YamlHelper(LoadSettings loadSettings, String yamlContent) {
this.loadSettings = loadSettings;
this.yamlContent = yamlContent;
}
// Constructor that reads YAML from a file path
public YamlHelper(Path originalFilePath) throws IOException {
this.yamlContent = Files.readString(originalFilePath);
this.originalFilePath = originalFilePath;
}
/**
* Updates values in the target YAML based on values from the source YAML. It ensures that only
* existing keys in the target YAML are updated.
*
* @return true if at least one key was updated, false otherwise.
*/
public boolean updateValuesFromYaml(YamlHelper sourceYaml, YamlHelper targetYaml) {
boolean updated = false;
Set<String> sourceKeys = sourceYaml.getAllKeys();
Set<String> targetKeys = targetYaml.getAllKeys();
for (String key : sourceKeys) {
String[] keyArray = key.split("\\.");
Object newValue = sourceYaml.getValueByExactKeyPath(keyArray);
Object currentValue = targetYaml.getValueByExactKeyPath(keyArray);
if (newValue != null
&& (!newValue.equals(currentValue) || !sourceKeys.equals(targetKeys))) {
boolean updatedKey = targetYaml.updateValue(Arrays.asList(keyArray), newValue);
if (updatedKey) updated = true;
}
}
return updated;
}
/**
* Updates a value in the YAML structure.
*
* @param keys The hierarchical keys leading to the value.
* @param newValue The new value to set.
* @return true if the value was updated, false otherwise.
*/
public boolean updateValue(List<String> keys, Object newValue) {
return updateValue(getRootNode(), keys, newValue);
}
private boolean updateValue(Node node, List<String> keys, Object newValue) {
if (!(node instanceof MappingNode mappingNode)) return false;
List<NodeTuple> updatedTuples = new ArrayList<>();
boolean updated = false;
for (NodeTuple tuple : mappingNode.getValue()) {
ScalarNode keyNode = (tuple.getKeyNode() instanceof ScalarNode sk) ? sk : null;
if (keyNode == null || !keyNode.getValue().equals(keys.get(0))) {
updatedTuples.add(tuple);
continue;
}
Node valueNode = tuple.getValueNode();
if (keys.size() == 1) {
Tag tag = valueNode.getTag();
Node newValueNode = null;
if (isAnyInteger(newValue)) {
newValueNode =
new ScalarNode(Tag.INT, String.valueOf(newValue), ScalarStyle.PLAIN);
} else if (isFloat(newValue)) {
Object floatValue = Float.valueOf(String.valueOf(newValue));
newValueNode =
new ScalarNode(
Tag.FLOAT, String.valueOf(floatValue), ScalarStyle.PLAIN);
} else if ("true".equals(newValue) || "false".equals(newValue)) {
newValueNode =
new ScalarNode(Tag.BOOL, String.valueOf(newValue), ScalarStyle.PLAIN);
} else if (newValue instanceof List<?> list) {
List<Node> sequenceNodes = new ArrayList<>();
for (Object item : list) {
Object obj = String.valueOf(item);
if (isAnyInteger(item)) {
tag = Tag.INT;
} else if (isFloat(item)) {
obj = Float.valueOf(String.valueOf(item));
tag = Tag.FLOAT;
} else if ("true".equals(item) || "false".equals(item)) {
tag = Tag.BOOL;
} else if (item == null || "null".equals(item)) {
tag = Tag.NULL;
} else {
tag = Tag.STR;
}
sequenceNodes.add(
new ScalarNode(tag, String.valueOf(obj), ScalarStyle.PLAIN));
}
newValueNode = new SequenceNode(Tag.SEQ, sequenceNodes, FlowStyle.FLOW);
} else if (tag == Tag.NULL) {
if ("true".equals(newValue)
|| "false".equals(newValue)
|| newValue instanceof Boolean) {
tag = Tag.BOOL;
}
newValueNode = new ScalarNode(tag, String.valueOf(newValue), ScalarStyle.PLAIN);
} else {
newValueNode = new ScalarNode(tag, String.valueOf(newValue), ScalarStyle.PLAIN);
}
copyComments(valueNode, newValueNode);
updatedTuples.add(new NodeTuple(keyNode, newValueNode));
updated = true;
} else if (valueNode instanceof MappingNode) {
updated = updateValue(valueNode, keys.subList(1, keys.size()), newValue);
updatedTuples.add(tuple);
}
}
if (updated) {
mappingNode.getValue().clear();
mappingNode.getValue().addAll(updatedTuples);
}
setNewNode(node);
return updated;
}
/**
* Fetches a value based on an exact key path.
*
* @param keys The key hierarchy leading to the value.
* @return The value if found, otherwise null.
*/
public Object getValueByExactKeyPath(String... keys) {
return getValueByExactKeyPath(getRootNode(), new ArrayDeque<>(List.of(keys)));
}
private Object getValueByExactKeyPath(Node node, Deque<String> keyQueue) {
if (!(node instanceof MappingNode mappingNode)) return null;
String currentKey = keyQueue.poll();
if (currentKey == null) return null;
for (NodeTuple tuple : mappingNode.getValue()) {
if (tuple.getKeyNode() instanceof ScalarNode keyNode
&& keyNode.getValue().equals(currentKey)) {
if (keyQueue.isEmpty()) {
Node valueNode = tuple.getValueNode();
if (valueNode instanceof ScalarNode scalarValueNode) {
return scalarValueNode.getValue();
} else if (valueNode instanceof MappingNode subMapping) {
return getValueByExactKeyPath(subMapping, keyQueue);
} else if (valueNode instanceof SequenceNode sequenceNode) {
List<Object> valuesList = new ArrayList<>();
for (Node o : sequenceNode.getValue()) {
if (o instanceof ScalarNode scalarValue) {
valuesList.add(scalarValue.getValue());
}
}
return valuesList;
} else {
return null;
}
}
return getValueByExactKeyPath(tuple.getValueNode(), keyQueue);
}
}
return null;
}
private Set<String> cachedKeys;
/**
* Retrieves the set of all keys present in the YAML structure. Keys are returned as
* dot-separated paths for nested keys.
*
* @return A set containing all keys in dot notation.
*/
public Set<String> getAllKeys() {
if (cachedKeys == null) {
cachedKeys = getAllKeys(getRootNode());
}
return cachedKeys;
}
/**
* Collects all keys from the YAML node recursively.
*
* @param node The current YAML node.
* @param currentPath The accumulated path of keys.
* @param allKeys The set storing all collected keys.
*/
private Set<String> getAllKeys(Node node) {
Set<String> allKeys = new LinkedHashSet<>();
collectKeys(node, "", allKeys);
return allKeys;
}
/**
* Recursively traverses the YAML structure to collect all keys.
*
* @param node The current node in the YAML structure.
* @param currentPath The accumulated key path.
* @param allKeys The set storing collected keys.
*/
private void collectKeys(Node node, String currentPath, Set<String> allKeys) {
if (node instanceof MappingNode mappingNode) {
for (NodeTuple tuple : mappingNode.getValue()) {
if (tuple.getKeyNode() instanceof ScalarNode keyNode) {
String newPath =
currentPath.isEmpty()
? keyNode.getValue()
: currentPath + "." + keyNode.getValue();
allKeys.add(newPath);
collectKeys(tuple.getValueNode(), newPath, allKeys);
}
}
}
}
/**
* Retrieves the root node of the YAML document. If a new node was previously set, it is
* returned instead.
*
* @return The root node of the YAML structure.
*/
private Node getRootNode() {
if (this.updatedRootNode != null) {
return this.updatedRootNode;
}
Composer composer = new Composer(loadSettings, getParserImpl());
Optional<Node> rootNodeOpt = composer.getSingleNode();
if (rootNodeOpt.isPresent()) {
return rootNodeOpt.get();
}
return null;
}
/**
* Sets a new root node, allowing modifications to be tracked.
*
* @param newRootNode The modified root node.
*/
public void setNewNode(Node newRootNode) {
this.updatedRootNode = newRootNode;
}
/**
* Retrieves the current root node (either the original or the updated one).
*
* @return The root node.
*/
public Node getUpdatedRootNode() {
if (this.updatedRootNode == null) {
this.updatedRootNode = getRootNode();
}
return this.updatedRootNode;
}
/**
* Initializes the YAML parser.
*
* @return The configured parser.
*/
private ParserImpl getParserImpl() {
return new ParserImpl(loadSettings, getStreamReader());
}
/**
* Creates a stream reader for the YAML content.
*
* @return The configured stream reader.
*/
private StreamReader getStreamReader() {
return new StreamReader(loadSettings, yamlContent);
}
public MappingNode save(Path saveFilePath) throws IOException {
if (!saveFilePath.equals(originalFilePath)) {
Files.writeString(saveFilePath, convertNodeToYaml(getUpdatedRootNode()));
}
return (MappingNode) getUpdatedRootNode();
}
public void saveOverride(Path saveFilePath) throws IOException {
Files.writeString(saveFilePath, convertNodeToYaml(getUpdatedRootNode()));
}
/**
* Converts a YAML node back to a YAML-formatted string.
*
* @param rootNode The root node to be converted.
* @return A YAML-formatted string.
*/
public String convertNodeToYaml(Node rootNode) {
StringWriter writer = new StringWriter();
StreamDataWriter streamDataWriter =
new StreamDataWriter() {
@Override
public void write(String str) {
writer.write(str);
}
@Override
public void write(String str, int off, int len) {
writer.write(str, off, len);
}
};
new Dump(DUMP_SETTINGS).dumpNode(rootNode, streamDataWriter);
return writer.toString();
}
private static boolean isParsable(String value, Function<String, ?> parser) {
try {
parser.apply(value);
return true;
} catch (NumberFormatException e) {
return false;
}
}
/**
* Checks if a given object is an integer.
*
* @param object The object to check.
* @return True if the object represents an integer, false otherwise.
*/
@SuppressWarnings("UnnecessaryTemporaryOnConversionFromString")
public static boolean isInteger(Object object) {
if (object instanceof Integer
|| object instanceof Short
|| object instanceof Byte
|| object instanceof Long) {
return true;
}
if (object instanceof String str) {
return isParsable(str, Integer::parseInt);
}
return false;
}
/**
* Checks if a given object is a floating-point number.
*
* @param object The object to check.
* @return True if the object represents a float, false otherwise.
*/
@SuppressWarnings("UnnecessaryTemporaryOnConversionFromString")
public static boolean isFloat(Object object) {
return (object instanceof Float || object instanceof Double)
|| (object instanceof String str && isParsable(str, Float::parseFloat));
}
/**
* Checks if a given object is a short integer.
*
* @param object The object to check.
* @return True if the object represents a short integer, false otherwise.
*/
@SuppressWarnings("UnnecessaryTemporaryOnConversionFromString")
public static boolean isShort(Object object) {
return (object instanceof Long)
|| (object instanceof String str && isParsable(str, Short::parseShort));
}
/**
* Checks if a given object is a byte.
*
* @param object The object to check.
* @return True if the object represents a byte, false otherwise.
*/
@SuppressWarnings("UnnecessaryTemporaryOnConversionFromString")
public static boolean isByte(Object object) {
return (object instanceof Long)
|| (object instanceof String str && isParsable(str, Byte::parseByte));
}
/**
* Checks if a given object is a long integer.
*
* @param object The object to check.
* @return True if the object represents a long integer, false otherwise.
*/
@SuppressWarnings("UnnecessaryTemporaryOnConversionFromString")
public static boolean isLong(Object object) {
return (object instanceof Long)
|| (object instanceof String str && isParsable(str, Long::parseLong));
}
/**
* Determines if an object is any type of integer (short, byte, long, or int).
*
* @param object The object to check.
* @return True if the object represents an integer type, false otherwise.
*/
public static boolean isAnyInteger(Object object) {
return isInteger(object) || isShort(object) || isByte(object) || isLong(object);
}
/**
* Copies comments from an old node to a new one.
*
* @param oldNode The original node with comments.
* @param newValueNode The new node to which comments should be copied.
*/
private void copyComments(Node oldNode, Node newValueNode) {
if (oldNode == null || newValueNode == null) return;
if (oldNode.getBlockComments() != null) {
newValueNode.setBlockComments(oldNode.getBlockComments());
}
if (oldNode.getInLineComments() != null) {
newValueNode.setInLineComments(oldNode.getInLineComments());
}
if (oldNode.getEndComments() != null) {
newValueNode.setEndComments(oldNode.getEndComments());
}
}
}

View File

@@ -31,14 +31,14 @@ public class SettingsController {
@PostMapping("/update-enable-analytics")
@Hidden
public ResponseEntity<String> updateApiKey(@RequestBody Boolean enabled) throws IOException {
if (!"undefined".equals(applicationProperties.getSystem().getEnableAnalytics())) {
if (applicationProperties.getSystem().getEnableAnalytics() != null) {
return ResponseEntity.status(HttpStatus.ALREADY_REPORTED)
.body(
"Setting has already been set, To adjust please edit "
+ InstallationPathConfig.getSettingsPath());
}
GeneralUtils.saveKeyToConfig("system.enableAnalytics", String.valueOf(enabled), false);
applicationProperties.getSystem().setEnableAnalytics(String.valueOf(enabled));
GeneralUtils.saveKeyToSettings("system.enableAnalytics", enabled);
applicationProperties.getSystem().setEnableAnalytics(enabled);
return ResponseEntity.ok("Updated");
}
}

View File

@@ -284,10 +284,14 @@ public class ApplicationProperties {
private boolean customHTMLFiles;
private String tessdataDir;
private Boolean enableAlphaFunctionality;
private String enableAnalytics;
private Boolean enableAnalytics;
private Datasource datasource;
private Boolean disableSanitize;
private CustomPaths customPaths = new CustomPaths();
public boolean isAnalyticsEnabled() {
return this.getEnableAnalytics() != null && this.getEnableAnalytics();
}
}
@Data

View File

@@ -49,7 +49,7 @@ public class PostHogService {
}
private void captureSystemInfo() {
if (!Boolean.parseBoolean(applicationProperties.getSystem().getEnableAnalytics())) {
if (!applicationProperties.getSystem().isAnalyticsEnabled()) {
return;
}
try {
@@ -60,7 +60,7 @@ public class PostHogService {
}
public void captureEvent(String eventName, Map<String, Object> properties) {
if (!Boolean.parseBoolean(applicationProperties.getSystem().getEnableAnalytics())) {
if (!applicationProperties.getSystem().isAnalyticsEnabled()) {
return;
}
postHog.capture(uniqueId, eventName, properties);
@@ -315,7 +315,7 @@ public class PostHogService {
addIfNotEmpty(
properties,
"system_enableAnalytics",
applicationProperties.getSystem().getEnableAnalytics());
applicationProperties.getSystem().isAnalyticsEnabled());
// Capture UI properties
addIfNotEmpty(properties, "ui_appName", applicationProperties.getUi().getAppName());

View File

@@ -9,15 +9,10 @@ import java.nio.charset.StandardCharsets;
import java.nio.file.*;
import java.nio.file.attribute.BasicFileAttributes;
import java.security.MessageDigest;
import java.util.ArrayDeque;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.Deque;
import java.util.Enumeration;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.UUID;
import org.springframework.web.multipart.MultipartFile;
@@ -30,6 +25,7 @@ import io.github.pixee.security.Urls;
import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.config.InstallationPathConfig;
import stirling.software.SPDF.config.YamlHelper;
@Slf4j
public class GeneralUtils {
@@ -338,218 +334,16 @@ public class GeneralUtils {
}
}
public static void saveKeyToConfig(String id, String key) throws IOException {
saveKeyToConfig(id, key, true);
}
public static void saveKeyToConfig(String id, boolean key) throws IOException {
saveKeyToConfig(id, key, true);
}
public static void saveKeyToConfig(String id, String key, boolean autoGenerated)
throws IOException {
doSaveKeyToConfig(id, (key == null ? "" : key), autoGenerated);
}
public static void saveKeyToConfig(String id, boolean key, boolean autoGenerated)
throws IOException {
doSaveKeyToConfig(id, String.valueOf(key), autoGenerated);
}
/*------------------------------------------------------------------------*
* Internal Implementation Details *
*------------------------------------------------------------------------*/
/**
* Actually performs the line-based update for the given path (e.g. "security.csrfDisabled") to
* a new string value (e.g. "true"), possibly marking it as auto-generated.
*/
private static void doSaveKeyToConfig(String fullPath, String newValue, boolean autoGenerated)
throws IOException {
// 1) Load the file (settings.yml)
public static void saveKeyToSettings(String key, Object newValue) throws IOException {
String[] keyArray = key.split("\\.");
Path settingsPath = Paths.get(InstallationPathConfig.getSettingsPath());
if (!Files.exists(settingsPath)) {
log.warn("Settings file not found at {}, creating a new empty file...", settingsPath);
Files.createDirectories(settingsPath.getParent());
Files.createFile(settingsPath);
}
List<String> lines = Files.readAllLines(settingsPath);
// 2) Build a map of "nestedKeyPath -> lineIndex" by parsing indentation
// Also track each line's indentation so we can preserve it when rewriting.
Map<String, LineInfo> pathToLine = parseNestedYamlKeys(lines);
// 3) If the path is found, rewrite its line. Else, append at the bottom (no indentation).
boolean changed = false;
if (pathToLine.containsKey(fullPath)) {
// Rewrite existing line
LineInfo info = pathToLine.get(fullPath);
String oldLine = lines.get(info.lineIndex);
String newLine =
rewriteLine(oldLine, info.indentSpaces, fullPath, newValue, autoGenerated);
if (!newLine.equals(oldLine)) {
lines.set(info.lineIndex, newLine);
changed = true;
}
} else {
// Append a new line at the bottom, with zero indentation
String appended = fullPath + ": " + newValue;
if (autoGenerated) {
appended += " # Automatically Generated Settings (Do Not Edit Directly)";
}
lines.add(appended);
changed = true;
}
// 4) If changed, write back to file
if (changed) {
Files.write(settingsPath, lines);
log.info(
"Updated '{}' to '{}' (autoGenerated={}) in {}",
fullPath,
newValue,
autoGenerated,
settingsPath);
} else {
log.info("No changes for '{}' (already set to '{}').", fullPath, newValue);
}
}
/** A small record-like class that holds: - lineIndex - indentSpaces */
private static class LineInfo {
int lineIndex;
int indentSpaces;
public LineInfo(int lineIndex, int indentSpaces) {
this.lineIndex = lineIndex;
this.indentSpaces = indentSpaces;
}
}
/**
* Parse the YAML lines to build a map: "full.nested.key" -> (lineIndex, indentSpaces). We do a
* naive indentation-based path stacking: - 2 spaces = 1 indent level - lines that start with
* fewer or equal indentation pop the stack - lines that look like "key:" or "key: value" cause
* a push
*/
private static Map<String, LineInfo> parseNestedYamlKeys(List<String> lines) {
Map<String, LineInfo> result = new HashMap<>();
// We'll maintain a stack of (keyName, indentLevel).
// Each line that looks like "myKey:" or "myKey: value" is a new "child" of the top of the
// stack if indent is deeper.
Deque<String> pathStack = new ArrayDeque<>();
Deque<Integer> indentStack = new ArrayDeque<>();
indentStack.push(-1); // sentinel
for (int i = 0; i < lines.size(); i++) {
String line = lines.get(i);
String trimmed = line.trim();
// skip blank lines, comment lines, or list items
if (trimmed.isEmpty() || trimmed.startsWith("#") || trimmed.startsWith("-")) {
continue;
}
// check if there's a colon
int colonIdx = trimmed.indexOf(':');
if (colonIdx <= 0) { // must have at least one char before ':'
continue;
}
// parse out key
String keyPart = trimmed.substring(0, colonIdx).trim();
if (keyPart.isEmpty()) {
continue;
}
// count leading spaces for indentation
int leadingSpaces = countLeadingSpaces(line);
int indentLevel = leadingSpaces / 2; // assume 2 spaces per level
// pop from stack until we get to a shallower indentation
while (indentStack.peek() != null && indentStack.peek() >= indentLevel) {
indentStack.pop();
pathStack.pop();
}
// push the new key
pathStack.push(keyPart);
indentStack.push(indentLevel);
// build the full path
String[] arr = pathStack.toArray(new String[0]);
List<String> reversed = Arrays.asList(arr);
Collections.reverse(reversed);
String fullPath = String.join(".", reversed);
// store line info
result.put(fullPath, new LineInfo(i, leadingSpaces));
}
return result;
}
/**
* Rewrite a single line to set a new value, preserving indentation and (optionally) the
* existing or auto-generated inline comment.
*
* <p>For example, oldLine might be: " csrfDisabled: false # set to 'true' to disable CSRF
* protection" newValue = "true" autoGenerated = false
*
* <p>We'll produce something like: " csrfDisabled: true # set to 'true' to disable CSRF
* protection"
*/
private static String rewriteLine(
String oldLine, int indentSpaces, String path, String newValue, boolean autoGenerated) {
// We'll keep the exact leading indentation (indentSpaces).
// Then "key: newValue". We'll try to preserve any existing inline comment unless
// autoGenerated is true.
// 1) Extract leading spaces from the old line (just in case they differ from indentSpaces).
int actualLeadingSpaces = countLeadingSpaces(oldLine);
String leading = oldLine.substring(0, actualLeadingSpaces);
// 2) Remove leading spaces from the rest
String trimmed = oldLine.substring(actualLeadingSpaces);
// 3) Check for existing comment
int hashIndex = trimmed.indexOf('#');
String lineWithoutComment =
(hashIndex >= 0) ? trimmed.substring(0, hashIndex).trim() : trimmed.trim();
String oldComment = (hashIndex >= 0) ? trimmed.substring(hashIndex).trim() : "";
// 4) Rebuild "key: newValue"
// The "key" here is everything before ':' in lineWithoutComment
int colonIdx = lineWithoutComment.indexOf(':');
String existingKey =
(colonIdx >= 0)
? lineWithoutComment.substring(0, colonIdx).trim()
: path; // fallback if line is malformed
StringBuilder sb = new StringBuilder();
sb.append(leading); // restore original leading spaces
// "key: newValue"
sb.append(existingKey).append(": ").append(newValue);
// 5) If autoGenerated, add/replace comment
if (autoGenerated) {
sb.append(" # Automatically Generated Settings (Do Not Edit Directly)");
} else {
// preserve the old comment if it exists
if (!oldComment.isEmpty()) {
sb.append(" ").append(oldComment);
}
}
return sb.toString();
}
private static int countLeadingSpaces(String line) {
int count = 0;
for (char c : line.toCharArray()) {
if (c == ' ') count++;
else break;
}
return count;
YamlHelper settingsYaml = new YamlHelper(settingsPath);
settingsYaml.updateValue(Arrays.asList(keyArray), newValue);
settingsYaml.saveOverride(settingsPath);
}
public static String generateMachineFingerprint() {