Building a Shared Ubuntu Desktop with VNC: A Developer’s Journey from Terminal to GUI

TL;DR

We successfully implemented a shared Ubuntu Desktop environment with VNC that allows our development team to collaboratively work in a GUI environment rather than just the command line. The solution uses XFCE desktop with x11vnc server, includes SSL encryption and secure authentication, and provides both native VNC client access and web-based access. This article details our implementation process, the challenges we faced, and how Claude helped us overcome technical hurdles.

The Problem: Terminal-Only Development Limitations

Our development environment had been terminal-only, which created several challenges for our team:

• Visual debugging required screen sharing through video conferencing tools
• Training new team members was cumbersome without a shared visual workspace
• Collaborative pair programming was difficult to coordinate
• Many useful GUI-based development and debugging tools couldn’t be utilized

We needed a solution that would provide a persistent graphical environment accessible to multiple team members simultaneously. After evaluating several options, we settled on implementing a shared Ubuntu Desktop accessible via VNC.

The VNC Desktop Implementation Timeline

Technical Implementation Details

System Architecture

Our VNC desktop solution consists of three main components working together:

┌─────────────────┐      ┌─────────────────┐      ┌─────────────────┐
│                 │      │                 │      │                 │
│   VNC Client    │─────▶│   VNC Server    │─────▶│  Ubuntu Desktop │
│   (Browser)     │      │   (x11vnc)      │      │   Environment   │
│                 │      │                 │      │                 │
└─────────────────┘      └─────────────────┘      └─────────────────┘

Phase 1: Base Installation

The first phase involved setting up the core desktop environment and VNC server. We chose the lightweight XFCE desktop for its balance of features and performance:

# Install XFCE desktop environment
sudo apt update
sudo apt install -y xfce4 xfce4-goodies

# Install x11vnc server
sudo apt install -y x11vnc

We selected x11vnc over alternatives like TightVNC or TigerVNC because it works with an existing X server session rather than creating a new one, which allows for true session sharing.

Phase 2: Security Hardening

Security was a primary concern, so we implemented multiple layers of protection:

# Create a password for VNC access
mkdir -p ~/.vnc
x11vnc -storepasswd YOUR_PASSWORD ~/.vnc/passwd

# Generate self-signed SSL certificate for encrypted connections
openssl req -x509 -newkey rsa:4096 -keyout ~/.vnc/x11vnc.key \
  -out ~/.vnc/x11vnc.cert -days 365 -nodes

We also configured the system firewall to restrict access to VNC ports and implemented detailed connection logging to track usage.

Phase 3: Service Configuration

To ensure the VNC service started automatically on system boot and recovered from crashes, we created a systemd service:

# Create startup script
cat > /usr/local/bin/start-vnc.sh << 'EOF'
#!/bin/bash

# Start the desktop environment if not already running
if ! pgrep xfce4-session > /dev/null; then
  startxfce4 &
  sleep 5
fi

# Start x11vnc with security options
x11vnc -display :0 -rfbauth ~/.vnc/passwd -ssl ~/.vnc/x11vnc.cert,~/.vnc/x11vnc.key \
  -forever -shared -bg -o ~/.vnc/x11vnc.log
EOF

chmod +x /usr/local/bin/start-vnc.sh

# Create systemd service
cat > /etc/systemd/system/x11vnc.service << 'EOF'
[Unit]
Description=X11VNC Service
After=display-manager.service network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/start-vnc.sh
Restart=on-failure
User=root

[Install]
WantedBy=multi-user.target
EOF

# Enable and start the service
systemctl enable x11vnc.service
systemctl start x11vnc.service

Phase 4: Web Access Implementation

To provide browser-based access without requiring VNC client installation, we set up noVNC:

# Install dependencies
sudo apt install -y git python3-pip

# Clone and set up noVNC
git clone https://github.com/novnc/noVNC.git
cd noVNC

# Create a startup script for noVNC
cat > start-novnc.sh << 'EOF'
#!/bin/bash
./utils/launch.sh --vnc localhost:5900 --listen 6080
EOF

chmod +x start-novnc.sh

# Create a systemd service for noVNC
cat > /etc/systemd/system/novnc.service << 'EOF'
[Unit]
Description=noVNC Service
After=x11vnc.service
Requires=x11vnc.service

[Service]
Type=simple
ExecStart=/path/to/noVNC/start-novnc.sh
Restart=on-failure
User=root
WorkingDirectory=/path/to/noVNC

[Install]
WantedBy=multi-user.target
EOF

# Enable and start the service
systemctl enable novnc.service
systemctl start novnc.service

Challenges We Faced

Display Resolution Issues

One of our biggest challenges was configuring a display resolution that worked well for all team members with different screen sizes. We initially set a fixed resolution, but this created problems for users with smaller or larger screens.

The solution was to implement dynamic resolution adjustment using x11vnc's scaling features:

x11vnc -display :0 -rfbauth ~/.vnc/passwd -ssl ~/.vnc/x11vnc.cert,~/.vnc/x11vnc.key \
  -forever -shared -bg -scale_cursor 1 -scale 1 -autoport 5900 -o ~/.vnc/x11vnc.log

Performance Issues with Multiple Connections

When multiple team members connected simultaneously, we noticed performance degradation. After investigation, we determined this was primarily due to bandwidth limitations rather than server CPU constraints.

We enhanced performance by implementing compression and encoding optimizations:

x11vnc -display :0 -rfbauth ~/.vnc/passwd -ssl ~/.vnc/x11vnc.cert,~/.vnc/x11vnc.key \
  -forever -shared -bg -scale_cursor 1 -scale 1 -autoport 5900 \
  -threads -ncache 10 -ncache_cr -tight -jpeg -compresslevel 9 \
  -o ~/.vnc/x11vnc.log

Session Management Challenges

We wanted to ensure that sessions remained persistent even when all users disconnected. This required careful configuration of the X server and desktop environment:

# Create a persistent X session on startup
cat > /etc/X11/xinit/xinitrc << 'EOF'
#!/bin/sh
exec startxfce4
EOF

# Configure XFCE power management to prevent sleeping/screen blanking
mkdir -p /etc/xdg/xfce4/xfconf/xfce-perchannel-xml/
cat > /etc/xdg/xfce4/xfconf/xfce-perchannel-xml/xfce4-power-manager.xml << 'EOF'


  
    
    
    
    
    
    
    
  

EOF

How Claude Helped Us

Claude was instrumental in our VNC implementation process. Here are specific examples of how Claude helped:

Debugging X Server Configuration Issues

When we faced issues with X server initialization, Claude helped identify the root cause and provided the correct solution:

Error: unable to open display ":0"

Claude identified that we needed to set the DISPLAY environment variable and ensure the X server was running before attempting to start VNC:

# Update start-vnc.sh to handle X server initialization
if ! xdpyinfo &>/dev/null; then
  export DISPLAY=:0
  Xvfb :0 -screen 0 1920x1080x24 &
  sleep 2
  startxfce4 &
  sleep 5
fi

Optimizing VNC Configuration

Claude helped us understand the performance implications of different VNC settings and recommended optimal configurations:

Security Configuration

Claude provided guidance on proper security configuration for both VNC and noVNC, including:

• Generating properly formatted SSL certificates
• Configuring secure password storage
• Setting up SSH tunneling for encrypted connections
• Implementing proper firewall rules

Results and Benefits

Our VNC desktop implementation has been transformative for our development process:

Quantitative Benefits

Qualitative Benefits

• Improved Collaboration: Team members can easily view and interact with the same desktop
• Enhanced Debugging: Visual debugging tools like browser inspectors are now readily available
• Better Training: New team members can observe experienced developers in real-time
• Expanded Toolset: Access to GUI-based development tools that weren't previously usable
• Reduced Context Switching: Less need to switch between terminal and graphical applications

Usage Guidelines for Team Members

Connecting with Native VNC Clients

For optimal performance, we recommend using native VNC clients:

• Linux: Remmina, Vinagre
• macOS: Screen Sharing (built-in), RealVNC Viewer
• Windows: TightVNC Viewer, RealVNC Viewer

Connection details:

• Server: [your-server-address]
• Port: 5900
• Password: [stored in password manager]
• Encryption: Enable SSL/TLS

Web-Based Access

For situations where installing a VNC client isn't possible:

1. Navigate to https://[your-server-address]:6080/vnc.html
2. Enter the VNC password when prompted
3. Use the browser-based interface

SSH Tunneling for Enhanced Security

For accessing the VNC server from untrusted networks, we recommend using SSH tunneling:

# Create SSH tunnel
ssh -L 5901:localhost:5900 user@[your-server-address]

# Then connect to localhost:5901 in your VNC client

Maintenance and Monitoring

We've implemented several measures to ensure the VNC service remains reliable:

• Automated Checks: Hourly service status verification
• Log Rotation: VNC logs are rotated daily to prevent disk space issues
• Performance Monitoring: Regular checks of CPU, memory, and network usage
• Backup Procedures: Daily backups of configuration files

# Monitor VNC service
cat > /usr/local/bin/check-vnc.sh << 'EOF'
#!/bin/bash

if ! pgrep x11vnc > /dev/null; then
  systemctl restart x11vnc.service
  echo "$(date): VNC service restarted" >> /var/log/vnc-monitor.log
fi
EOF

chmod +x /usr/local/bin/check-vnc.sh

# Add to crontab
(crontab -l 2>/dev/null; echo "0 * * * * /usr/local/bin/check-vnc.sh") | crontab -

Future Enhancements

While our current implementation meets our needs, we've identified several potential improvements for the future:

• User Authentication Integration: Connect VNC authentication with our team's identity management system
• Session Recording: Implement recording capabilities for training and documentation purposes
• Multiple Desktops: Create separate desktop environments for different projects or teams
• Performance Optimizations: Further tuning for specific types of visual content
• Mobile Access: Optimize the web interface for tablet and smartphone access

Conclusion

Our journey from a terminal-only environment to a full GUI desktop has dramatically improved our development workflow. By implementing a shared VNC desktop solution, we've enabled better collaboration, enhanced our debugging capabilities, and expanded our toolset beyond command-line utilities.

The combination of XFCE desktop, x11vnc server, and noVNC web access provides a flexible, secure, and performant solution that meets the needs of our development team. With multiple security layers, performance optimizations, and comprehensive documentation, we've created a system that will serve us well into the future.

For teams considering a similar implementation, our experience demonstrates that the benefits far outweigh the initial setup effort. The productivity gains from improved collaboration and expanded tooling make this a worthwhile investment for any development team currently limited to terminal-only access.