655 lines
18 KiB
Markdown
655 lines
18 KiB
Markdown
|
|
## **6. Improving the `extract_title` Function**
|
|
|
|
### **a. Handling Complex YAML Frontmatter**
|
|
|
|
**Issue:** YAML frontmatter can contain more complex structures, including quotes, multiple spaces, or special characters, which the current `extract_title` function might not handle correctly.
|
|
|
|
**Solution:** Enhance the title extraction to handle various YAML frontmatter formats more robustly.
|
|
|
|
**Implementation:**
|
|
|
|
```bash
|
|
extract_title() {
|
|
local md_file="$1"
|
|
# Extract title from YAML frontmatter, handling quotes and spaces
|
|
local title
|
|
title=$(awk '/^title:/ {
|
|
sub(/^title:\s*/, "");
|
|
gsub(/['"'"']/,"", $0);
|
|
print
|
|
exit
|
|
}' "$md_file") || true
|
|
if [[ -z "$title" ]]; then
|
|
# Fallback to filename without extension
|
|
title=$(basename "$md_file" .md.old)
|
|
fi
|
|
echo "$title"
|
|
}
|
|
```
|
|
|
|
**Explanation:**
|
|
|
|
- **`awk` Usage:**
|
|
- Searches for a line starting with `title:`.
|
|
- Removes the `title:` prefix and any leading whitespace.
|
|
- Removes surrounding quotes if present.
|
|
- Prints the title and exits to prevent processing the entire file.
|
|
|
|
---
|
|
|
|
## **10. Incorporating Configuration Flexibility**
|
|
|
|
### **Issue:**
|
|
Hardcoding paths and settings reduces the script's flexibility and adaptability to different environments.
|
|
|
|
### **Solution:**
|
|
Allow configuration through external files or command-line arguments, making the script more versatile.
|
|
|
|
**Implementation:**
|
|
|
|
1. **Using a Configuration File:**
|
|
|
|
- **Create a Config File (`config.sh`):**
|
|
|
|
```bash
|
|
#!/usr/bin/env bash
|
|
|
|
SOURCE_DIR="$HOME/vimwiki"
|
|
TEMP_DIR="/tmp/vimwikihtml"
|
|
CSS_FILE="$HOME/.local/share/nvim/style.css"
|
|
CONCURRENT_JOBS=4
|
|
LOG_FILE="$TEMP_DIR/conversion.log"
|
|
ERROR_LOG_FILE="$TEMP_DIR/error.log"
|
|
```
|
|
|
|
- **Source the Config File in the Script:**
|
|
|
|
```bash
|
|
# At the beginning of test.sh
|
|
if [[ -f "$HOME/vimwiki/config.sh" ]]; then
|
|
source "$HOME/vimwiki/config.sh"
|
|
else
|
|
echo "Error: Configuration file 'config.sh' not found in '$HOME/vimwiki/'."
|
|
exit 1
|
|
fi
|
|
```
|
|
|
|
2. **Using Command-Line Arguments:**
|
|
|
|
- **Parse Arguments Using `getopts`:**
|
|
|
|
```bash
|
|
while getopts "s:t:c:j:l:e:" opt; do
|
|
case $opt in
|
|
s) SOURCE_DIR="$OPTARG" ;;
|
|
t) TEMP_DIR="$OPTARG" ;;
|
|
c) CSS_FILE="$OPTARG" ;;
|
|
j) CONCURRENT_JOBS="$OPTARG" ;;
|
|
l) LOG_FILE="$OPTARG" ;;
|
|
e) ERROR_LOG_FILE="$OPTARG" ;;
|
|
*) echo "Invalid option"; exit 1 ;;
|
|
esac
|
|
done
|
|
```
|
|
|
|
- **Usage Example:**
|
|
|
|
```bash
|
|
./test.sh -s "/path/to/source" -t "/path/to/temp" -c "/path/to/style.css" -j 8
|
|
```
|
|
|
|
**Benefits:**
|
|
|
|
- **Flexibility:** Easily adapt to different directories, styles, and settings without modifying the script.
|
|
- **Reusability:** Share the script across different projects with varying configurations.
|
|
|
|
---
|
|
|
|
## **11. Adding a Dry-Run Mode for Safe Testing**
|
|
|
|
### **Issue:**
|
|
Making changes to files (like deletions) without a preview can be risky, especially during testing.
|
|
|
|
### **Solution:**
|
|
Implement a dry-run mode that simulates actions without performing them, allowing you to verify behavior before actual execution.
|
|
|
|
**Implementation:**
|
|
|
|
1. **Introduce a `DRY_RUN` Variable:**
|
|
|
|
```bash
|
|
DRY_RUN=false
|
|
```
|
|
|
|
2. **Parse a Command-Line Argument for Dry-Run:**
|
|
|
|
```bash
|
|
while getopts "s:t:c:j:l:e:dr" opt; do
|
|
case $opt in
|
|
# ... [existing options]
|
|
d) DRY_RUN=true ;;
|
|
r) # Any other options
|
|
;;
|
|
*) echo "Invalid option"; exit 1 ;;
|
|
esac
|
|
done
|
|
```
|
|
|
|
3. **Modify File Operations to Respect Dry-Run:**
|
|
|
|
```bash
|
|
handle_deletions() {
|
|
echo "Handling deletions of Markdown files..."
|
|
|
|
find "$TEMP_DIR" -type f -name '*.md.old' -print0 | while IFS= read -r -d '' md_old_file; do
|
|
source_md="$SOURCE_DIR/${md_old_file#$TEMP_DIR/}"
|
|
source_md="${source_md%.md.old}.md"
|
|
|
|
if [[ ! -f "$source_md" ]]; then
|
|
html_file="${md_old_file%.md.old}.html"
|
|
|
|
if [[ "$DRY_RUN" = true ]]; then
|
|
echo "Dry-Run: Would delete '$html_file' as the source Markdown file no longer exists."
|
|
else
|
|
if [[ -f "$html_file" ]]; then
|
|
rm "$html_file"
|
|
echo "Deleted '$html_file' as the source Markdown file no longer exists."
|
|
echo "$(date +"%Y-%m-%d %H:%M:%S") - Deleted '$html_file' due to source removal." >> "$LOG_FILE"
|
|
fi
|
|
|
|
rm "$md_old_file"
|
|
echo "Removed obsolete '$md_old_file'."
|
|
fi
|
|
fi
|
|
done
|
|
|
|
echo "Deletion handling completed."
|
|
}
|
|
```
|
|
|
|
4. **Inform Users When in Dry-Run Mode:**
|
|
|
|
```bash
|
|
if [[ "$DRY_RUN" = true ]]; then
|
|
echo "Running in Dry-Run mode. No changes will be made."
|
|
fi
|
|
```
|
|
|
|
**Benefits:**
|
|
|
|
- **Safety:** Allows you to verify what actions the script would take without making actual changes.
|
|
- **Testing:** Facilitates safe testing and debugging of the script's logic.
|
|
|
|
---
|
|
|
|
|
|
**Explanation:**
|
|
|
|
- **`trap cleanup SIGINT SIGTERM`:** Catches interrupt (`Ctrl+C`) and termination signals, invoking the `cleanup` function.
|
|
- **`cleanup` Function:**
|
|
- Echoes a message indicating interruption.
|
|
- Terminates all background jobs to prevent orphan processes.
|
|
- Exits the script with a non-zero status.
|
|
|
|
---
|
|
|
|
## **13. Providing User Feedback and Progress Indicators**
|
|
|
|
### **Issue:**
|
|
With large numbers of files, users might find it hard to gauge progress or know if the script is still running.
|
|
|
|
### **Solution:**
|
|
Implement progress indicators or verbose outputs to inform users about the script's status.
|
|
|
|
**Implementation:**
|
|
|
|
1. **Using Progress Bars with `pv`:**
|
|
|
|
**Install `pv`:**
|
|
|
|
```bash
|
|
sudo apt-get install pv
|
|
```
|
|
|
|
2. **Integrate `pv` into File Processing:**
|
|
|
|
```bash
|
|
convert_all_md_to_html() {
|
|
echo "Starting conversion of updated Markdown files to HTML..."
|
|
|
|
find "$TEMP_DIR" -type f -name '*.md.old' -print0 | pv -0 -l -s "$(find "$TEMP_DIR" -type f -name '*.md.old' | wc -l)" | parallel -0 -j "$CONCURRENT_JOBS" convert_md_to_html {}
|
|
|
|
echo "Markdown to HTML conversion completed."
|
|
}
|
|
```
|
|
|
|
3. **Simple Progress Indicators:**
|
|
|
|
Alternatively, use echo statements to indicate progress.
|
|
|
|
```bash
|
|
convert_md_to_html() {
|
|
local md_old_file="$1"
|
|
echo "Processing file: $md_old_file"
|
|
# ... [rest of the function]
|
|
}
|
|
```
|
|
|
|
**Benefits:**
|
|
|
|
- **User Awareness:** Users are informed about the script's progress, enhancing usability.
|
|
- **Debugging:** Progress indicators can help identify if the script is stuck or processing unusually.
|
|
|
|
---
|
|
|
|
## **14. Adding a Help Message and Usage Instructions**
|
|
|
|
### **Issue:**
|
|
Users might not be aware of available options or how to use the script effectively.
|
|
|
|
### **Solution:**
|
|
Implement a help message that outlines usage instructions and available options.
|
|
|
|
**Implementation:**
|
|
|
|
```bash
|
|
print_help() {
|
|
echo "Usage: $0 [options]"
|
|
echo ""
|
|
echo "Options:"
|
|
echo " -s DIR Source Vimwiki directory (default: \$HOME/vimwiki)"
|
|
echo " -t DIR Temporary HTML output directory (default: /tmp/vimwikihtml)"
|
|
echo " -c FILE Path to the CSS file for styling (default: \$HOME/.local/share/nvim/style.css)"
|
|
echo " -j NUM Number of concurrent pandoc processes (default: 4)"
|
|
echo " -l FILE Log file to track conversions (default: /tmp/vimwikihtml/conversion.log)"
|
|
echo " -e FILE Log file to track errors (default: /tmp/vimwikihtml/error.log)"
|
|
echo " -d Enable Dry-Run mode"
|
|
echo " -h Display this help message"
|
|
echo ""
|
|
echo "Example:"
|
|
echo " $0 -s ~/myvimwiki -t ~/htmloutput -c ~/styles/style.css -j 8"
|
|
}
|
|
```
|
|
|
|
Add argument parsing to handle the `-h` option:
|
|
|
|
```bash
|
|
# Parse command-line arguments
|
|
while getopts "s:t:c:j:l:e:dh" opt; do
|
|
case $opt in
|
|
s) SOURCE_DIR="$OPTARG" ;;
|
|
t) TEMP_DIR="$OPTARG" ;;
|
|
c) CSS_FILE="$OPTARG" ;;
|
|
j) CONCURRENT_JOBS="$OPTARG" ;;
|
|
l) LOG_FILE="$OPTARG" ;;
|
|
e) ERROR_LOG_FILE="$OPTARG" ;;
|
|
d) DRY_RUN=true ;;
|
|
h)
|
|
print_help
|
|
exit 0
|
|
;;
|
|
*)
|
|
echo "Invalid option."
|
|
print_help
|
|
exit 1
|
|
;;
|
|
esac
|
|
done
|
|
```
|
|
|
|
**Benefits:**
|
|
|
|
- **Usability:** Makes the script more user-friendly and accessible.
|
|
- **Documentation:** Provides immediate reference without external documentation.
|
|
|
|
---
|
|
|
|
## **17. Adding Verbose and Quiet Modes**
|
|
|
|
### **Issue:**
|
|
Sometimes users might prefer minimal output, especially when running the script in automated environments, while other times they might want detailed logs.
|
|
|
|
### **Solution:**
|
|
Implement verbose (`-v`) and quiet (`-q`) modes to control the script's output.
|
|
|
|
**Implementation:**
|
|
|
|
1. **Introduce Variables:**
|
|
|
|
```bash
|
|
VERBOSE=false
|
|
QUIET=false
|
|
```
|
|
|
|
2. **Update Argument Parsing:**
|
|
|
|
```bash
|
|
while getopts "s:t:c:j:l:e:dhvq" opt; do
|
|
case $opt in
|
|
# ... [existing options]
|
|
v) VERBOSE=true ;;
|
|
q) QUIET=true ;;
|
|
# ... [other options]
|
|
esac
|
|
done
|
|
```
|
|
|
|
3. **Modify Echo Statements:**
|
|
|
|
```bash
|
|
log() {
|
|
if [[ "$VERBOSE" = true && "$QUIET" = false ]]; then
|
|
echo "$@"
|
|
fi
|
|
}
|
|
|
|
# Replace echo with log where appropriate
|
|
log "Starting conversion of '$md_old_file'..."
|
|
```
|
|
|
|
4. **Suppress Non-Essential Output in Quiet Mode:**
|
|
|
|
```bash
|
|
if [[ "$QUIET" = true ]]; then
|
|
exec >/dev/null 2>&1
|
|
fi
|
|
```
|
|
|
|
**Benefits:**
|
|
|
|
- **Flexibility:** Users can choose the level of output based on their needs.
|
|
- **Usability:** Enhances the script's adaptability in different contexts.
|
|
|
|
---
|
|
|
|
## **18. Documentation and In-Line Comments**
|
|
|
|
### **Issue:**
|
|
As scripts grow in complexity, maintaining clear documentation and comments becomes essential for future maintenance and collaboration.
|
|
|
|
### **Solution:**
|
|
Add comprehensive in-line comments explaining the purpose and functionality of code sections and functions.
|
|
|
|
**Implementation:**
|
|
|
|
```bash
|
|
# Function to convert a single Markdown file to HTML
|
|
# Arguments:
|
|
# $1 - Path to the .md.old file
|
|
convert_md_to_html() {
|
|
# ... [function code]
|
|
}
|
|
```
|
|
|
|
**Recommendation:** Regularly update comments to reflect any changes in the script's logic or functionality.
|
|
|
|
---
|
|
|
|
## **19. Testing and Validation**
|
|
|
|
### **Issue:**
|
|
Ensuring that the script behaves as expected in various scenarios is crucial for reliability.
|
|
|
|
### **Solution:**
|
|
Implement automated tests or conduct thorough manual testing covering different cases like:
|
|
|
|
- Adding new files.
|
|
- Modifying existing files.
|
|
- Deleting files.
|
|
- Handling files with special characters.
|
|
- Running in dry-run mode.
|
|
- Handling permission issues.
|
|
|
|
**Implementation:**
|
|
|
|
1. **Automated Testing:**
|
|
|
|
- **Use Bash Unit Testing Frameworks:** Tools like [Bats](https://github.com/bats-core/bats-core) can help write automated tests for your script.
|
|
|
|
- **Example Test Case:**
|
|
|
|
```bash
|
|
@test "Convert new Markdown file to HTML" {
|
|
# Setup: Create a new Markdown file in SOURCE_DIR
|
|
echo "# Test Page" > "$SOURCE_DIR/test.md"
|
|
|
|
# Run the script
|
|
~/vimwiki/test.sh
|
|
|
|
# Assert: Check if HTML file exists
|
|
[ -f "$TEMP_DIR/test.html" ]
|
|
|
|
# Cleanup
|
|
rm "$SOURCE_DIR/test.md" "$TEMP_DIR/test.html" "$TEMP_DIR/test.md.old"
|
|
}
|
|
```
|
|
|
|
2. **Manual Testing:**
|
|
|
|
- **Create Test Files:** Set up various Markdown files with different content and filenames.
|
|
- **Run the Script:** Execute the script and verify the HTML output.
|
|
- **Check Logs:** Ensure that logs accurately reflect the actions taken.
|
|
- **Edge Cases:** Test with empty files, very large files, and files with complex YAML frontmatter.
|
|
|
|
**Benefits:**
|
|
|
|
- **Reliability:** Ensures that the script performs correctly under various conditions.
|
|
- **Confidence:** Builds trust in the script's functionality before deploying it in critical environments.
|
|
|
|
---
|
|
|
|
## **20. Additional Feature Suggestions**
|
|
|
|
### **a. Watch Mode for Real-Time Updates**
|
|
|
|
**Issue:**
|
|
Manually running the script after every change can be tedious.
|
|
|
|
**Solution:**
|
|
Implement a watch mode that monitors the source directory for changes and triggers the script automatically.
|
|
|
|
**Implementation:**
|
|
|
|
1. **Use `inotifywait`:**
|
|
|
|
**Install `inotify-tools`:**
|
|
|
|
```bash
|
|
sudo apt-get install inotify-tools
|
|
```
|
|
|
|
2. **Add a Watch Mode Option:**
|
|
|
|
```bash
|
|
WATCH_MODE=false
|
|
|
|
while getopts "s:t:c:j:l:e:dhvqw" opt; do
|
|
case $opt in
|
|
# ... [existing options]
|
|
w) WATCH_MODE=true ;;
|
|
# ... [other options]
|
|
esac
|
|
done
|
|
```
|
|
|
|
3. **Implement Watch Mode in the Script:**
|
|
|
|
```bash
|
|
watch_mode() {
|
|
echo "Entering Watch Mode. Monitoring '$SOURCE_DIR' for changes..."
|
|
while inotifywait -r -e modify,create,delete,move "$SOURCE_DIR"; do
|
|
echo "Change detected. Updating Vimwiki HTML..."
|
|
update_if_needed
|
|
open_browser
|
|
done
|
|
}
|
|
|
|
main() {
|
|
check_dependencies
|
|
|
|
if [[ "$WATCH_MODE" = true ]]; then
|
|
# Initial synchronization
|
|
if [[ ! -d "$TEMP_DIR" ]]; then
|
|
mkdir -p "$TEMP_DIR"
|
|
synchronize_markdown
|
|
rename_md_files
|
|
handle_deletions
|
|
generate_index
|
|
else
|
|
update_if_needed
|
|
fi
|
|
|
|
# Enter watch mode
|
|
watch_mode
|
|
else
|
|
# Regular execution
|
|
if [[ ! -d "$TEMP_DIR" ]]; then
|
|
echo "Temporary directory '$TEMP_DIR' does not exist. Creating and performing full synchronization."
|
|
mkdir -p "$TEMP_DIR"
|
|
synchronize_markdown
|
|
rename_md_files
|
|
handle_deletions
|
|
generate_index
|
|
else
|
|
echo "Temporary directory '$TEMP_DIR' already exists. Checking for updates."
|
|
update_if_needed
|
|
fi
|
|
|
|
open_browser
|
|
|
|
echo "All tasks completed successfully."
|
|
fi
|
|
}
|
|
```
|
|
|
|
**Benefits:**
|
|
|
|
- **Convenience:** Automatically keeps the HTML output up-to-date in real-time.
|
|
- **Efficiency:** Eliminates the need for manual script execution after each change.
|
|
|
|
**Note:** Watch mode can consume more resources. Use it judiciously based on your system's capabilities.
|
|
|
|
---
|
|
|
|
## **22. Implementing a Configuration Validation Step**
|
|
|
|
### **Issue:**
|
|
Incorrect configurations (like wrong directory paths) can lead to script failures or unintended behavior.
|
|
|
|
### **Solution:**
|
|
Validate configuration parameters at the beginning of the script to ensure they are correct.
|
|
|
|
**Implementation:**
|
|
|
|
```bash
|
|
validate_config() {
|
|
# Validate SOURCE_DIR
|
|
if [[ ! -d "$SOURCE_DIR" ]]; then
|
|
handle_error "Source directory '$SOURCE_DIR' does not exist."
|
|
exit "$EXIT_FAILURE"
|
|
fi
|
|
|
|
# Validate CSS_FILE
|
|
if [[ ! -f "$CSS_FILE" ]]; then
|
|
echo "Warning: CSS file '$CSS_FILE' does not exist. HTML files will be generated without styling."
|
|
fi
|
|
|
|
# Validate TEMP_DIR (already handled in main)
|
|
}
|
|
|
|
main() {
|
|
check_dependencies
|
|
validate_config
|
|
|
|
# ... [rest of the main function]
|
|
}
|
|
```
|
|
|
|
**Benefits:**
|
|
|
|
- **Prevention:** Catches configuration issues early, preventing the script from running with invalid settings.
|
|
- **User Guidance:** Provides clear error messages to help users correct configurations.
|
|
|
|
---
|
|
|
|
## **23. Utilizing `set -x` for Debugging**
|
|
|
|
### **Issue:**
|
|
Debugging complex scripts can be challenging without visibility into their execution flow.
|
|
|
|
### **Solution:**
|
|
Use Bash's debugging options to trace the script's execution when needed.
|
|
|
|
**Implementation:**
|
|
|
|
1. **Add a Verbose Flag (`-x`):**
|
|
|
|
```bash
|
|
set -euo pipefail
|
|
[[ "$VERBOSE" = true ]] && set -x
|
|
```
|
|
|
|
2. **Enable Verbose Mode via Command-Line Argument:**
|
|
|
|
```bash
|
|
while getopts "s:t:c:j:l:e:dhvqx" opt; do
|
|
case $opt in
|
|
# ... [existing options]
|
|
x) VERBOSE=true ;;
|
|
# ... [other options]
|
|
esac
|
|
done
|
|
```
|
|
|
|
**Benefits:**
|
|
|
|
- **Transparency:** Provides detailed execution logs for debugging purposes.
|
|
- **Control:** Users can enable or disable debugging as needed without modifying the script.
|
|
|
|
**Caution:** Ensure that verbose mode does not expose sensitive information, especially when handling logs.
|
|
|
|
---
|
|
|
|
## **24. Preventing Infinite Loops or Excessive Resource Consumption**
|
|
|
|
### **Issue:**
|
|
If the script triggers file changes (like converting `.md.old` to `.html`), it might inadvertently cause itself to detect changes, leading to infinite loops or excessive resource usage.
|
|
|
|
### **Solution:**
|
|
Ensure that the script only processes intended file types and excludes its own output from triggering further actions.
|
|
|
|
**Implementation:**
|
|
|
|
1. **Exclude `.html` and `.md.old` Files from Synchronization:**
|
|
|
|
- Already handled via `rsync` excludes.
|
|
|
|
2. **Ensure `handle_deletions` and Other Functions Do Not Modify Source Directory:**
|
|
|
|
- All file operations target the temporary directory, ensuring the source remains untouched.
|
|
|
|
3. **Double-Check `rsync` Include/Exclude Patterns:**
|
|
|
|
```bash
|
|
rsync -av --delete \
|
|
--exclude='*.html' \
|
|
--exclude='*.sh' \
|
|
--exclude='.git/' \
|
|
--exclude='.gitignore' \
|
|
--exclude='*.bak' \
|
|
--include='*/' \
|
|
--include='*.md' \
|
|
--include='*.pdf' \
|
|
--include='*.png' \
|
|
--include='*.jpg' \
|
|
--include='*.jpeg' \
|
|
--include='*.gif' \
|
|
--exclude='*' \
|
|
"$SOURCE_DIR/" "$TEMP_DIR/"
|
|
```
|
|
|
|
**Recommendation:** Regularly review and test the script to ensure it doesn't inadvertently process or modify unintended files.
|
|
|
|
---
|