Skip to content

Troubleshooting

Please follow steps below if you encounter issues when using Windows Contrail.

1. Use contrail-status

Open the PowerShell command prompt (with the Administrator priviledge) on the compute node. Run contrail-status.ps1 from https://github.com/Juniper/contrail-windows/tree/master/scripts Make sure that the following services listed by the script are running: Docker, contrail-cnm-plugin, contrail-vrouter-nodemgr and contrail-vrouter-agent. Also make sure that the vrouter extension that is listed by the script is enabled and running. You may also run the script using the optional -Verbose option to get more details of the system as well as get the location of several important files such as logs etc.

2. Diagnostic check

Run Invoke-DiagnosticCheck.ps1 script from tools repository on Windows Computes that cause problems. Refer to README in root of the repository for script configuration info.

Note: the script can be ran with or without Administrator privileges. However, some checks will not be performed without them.

3. Microsoft's diagnostic check

Run Microsoft's Debug-ContainerHost.ps1 script:

Invoke-WebRequest https://aka.ms/Debug-ContainerHost.ps1 -UseBasicParsing | Invoke-Expression

For other problems with Windows Containers, consult the official troubleshooting docs.

4. Check the logs

Diagnostic check already looks for issues in logs of Contrail Components running on the node. However, you might want to check them manually.

Logs are located in $Env:ProgramData\contrail\var\logs\contrail\.

5. Verify the configuration

You might want to verify the configuration of all Contrail services running on the node.

Config files are located in $Env:ProgramData\contrail\etc\contrail\

6. Known issues

Search known issues section for symptoms of your issue. The section contains recovery procedures.

7. Use Contrail utility tools

Note: the following steps require experience using Contrail utils.

Behaviour of Windows Contrail utility tools is similar to their Linux counterparts. They are: vif, nh, rt, flow, mpls, vrfstats, dropstats, vxlan, vrouter, vrmemstats.

They should be in $Path. If not, they can be found under $Env:ProgramFiles\Juniper Networks\vRouter utilities\.

8. Crashdumps

If you encounter any kernel panics or service crashes, debugging dump files may prove useful:

  • Kernel crashdumps: $Env:SystemRoot\MEMORY.DMP and $Env:SystemRoot\Minidump\*dmp
  • Usermode crashdumps: $Env:LocalAppData\CrashDumps

9. PDB files

Debugging symbols for Contrail Components can be found in PDB files in Artifacts folder, next to MSI files. Artifacts folder is unpacked from docker containers on test beds and can be found directly on C: drive. For debugging you can use WinDbg or DbgShell (if you know PowerShell).

Warning: Remember, that every PDB file is bound to executable which was built with it (verified by checksum).

10. Total cleanup

Run Clear-ComputeNode.ps1 script from tools repository on Windows Computes that cause problems. Refer to README in root of the repository for script configuration info.

Warning: The script tries to clear all traces of Windows Contrail along with any leftover state. This includes any workload instances (containers).

Note: For ease of use, you can use Invoke-ScriptInRemoteSessions.ps1 script that will run the cleanup on a remote machine:

\Invoke-ScriptInRemoteSessions.ps1 -ScriptFileName ".\Clear-ComputeNode.ps1" -Addresses "<IP1>,<IP2>" -Credential (Get-Credential)

Redeployment of Windows Compute node is required after runnig this script. Please refer to deployment instructions.