YuzuZensai/netbird-kubernetes-operator
1
0
Fork 0
mirror of https://github.com/YuzuZensai/netbird-kubernetes-operator.git synced 2026-09-13 10:49:15 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update documentation with refactored resources (#204)

Browse Source 
This updates the documentation and examples to use the new resources. It
also adds automatic documentation creation for the api reference.

Signed-off-by: Philip Laine <philip.laine@gmail.com>
This commit is contained in:
Philip Laine
2026-04-24 16:22:04 +02:00
committed by GitHub
parent fe715ae40c
commit dd054d50ec
19 changed files with 642 additions and 464 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/usage.md
+94 -168
View File
@@ -1,210 +1,136 @@
# Usage
## Provision pods with NetBird access using side-cars
## Exposing Services
1. Create a Setup Key in your [NetBird console](https://docs.netbird.io/how-to/register-machines-using-setup-keys#using-setup-keys).
2. Create a Secret object in the namespace where you need to provision NetBird access (secret name and field can be anything).
The operator exposes Kubernetes services to your NetBird network by combining two resources, a `NetworkRouter` and a `NetworkResource`.
### NetworkRouter
A `NetworkRouter` creates a network in NetBird and deploys routing peer pods in the cluster. These pods are configured as routing peers for the network, handling traffic between NetBird clients and services running in the cluster.
Before creating a `NetworkRouter`, you must first create a custom DNS zone in the [NetBird dashboard](https://docs.netbird.io/manage/dns/custom-zones). The DNS zone must exist before the operator can register it.
```yaml
apiVersion: v1
stringData:
setupkey: "EEEEEEEE-EEEE-EEEE-EEEE-EEEEEEEEEEEE"
kind: Secret
apiVersion: netbird.io/v1alpha1
kind: NetworkRouter
metadata:
name: test
```
1. Create an NBSetupKey object referring to your secret.
```yaml
apiVersion: netbird.io/v1
kind: NBSetupKey
metadata:
name: test
name: prod
namespace: netbird
spec:
# Optional, overrides management URL for this setupkey only
# defaults to https://api.netbird.io
managementURL: https://netbird.example.com
secretKeyRef:
name: test # Required
key: setupkey # Required
```
1. Annotate the pods you need to inject NetBird into with `netbird.io/setup-key`.
```yaml
kind: Deployment
...
spec:
...
template:
metadata:
annotations:
netbird.io/setup-key: test # Must match the name of an NBSetupKey object in the same namespace
...
spec:
containers:
...
dnsZoneRef:
name: prod.company.internal
```
Since v0.27.0, NetBird supports extra DNS labels, which extends the DNS names that you can link to peers by grouping them and load balancing access using DNS round-robin. To enable this feature, add the following annotation to the pod:
```yaml
netbird.io/extra-dns-labels: "label1,label2"
```
With this setup, all peers with the same extra label would be used in a DNS round-robin fashion.
### NetworkResource
### Init Sidecar Mode
A `NetworkResource` exposes a Kubernetes service in NetBird by creating a matching resource in the routers network. The cluster IP of the service will be used as the resource IP. A record in the routers zone will also be created using the name and namespace of the service. The following example creates an nignx deployment and exposes the service with the record `nginx.default.prod.company.internal`.
By default, the NetBird container is injected as a regular sidecar container. For workloads like Jobs and CronJobs where the pod should terminate after the main container completes, you can use init sidecar mode. This injects NetBird as an init container with `restartPolicy: Always`.
To enable init sidecar mode, add the following annotation:
```yaml
netbird.io/init-sidecar: "true"
```
## Provisioning Networks (Ingress Functionality)
### Granting controller access to NetBird Management
> [!IMPORTANT]
> The NetBird Kubernetes operator generates configurations using NetBird API; editing or deleting these configurations in the NetBird console may cause temporary network disconnection until the operator reconciles the configuration.
1. Create a Service User on your NetBird dashboard (Must be Admin). [Doc](https://docs.netbird.io/how-to/access-netbird-public-api#creating-a-service-user).
2. Create an access token for the Service User (Must be Admin). [Doc](https://docs.netbird.io/how-to/access-netbird-public-api#creating-a-service-user).
3. Add access token to your helm values file under `netbirdAPI.key`.
1. Alternatively, provision secret in the same namespace as the operator and set the key `NB_API_KEY` to the access token generated.
2. Set `netbirdAPI.keyFromSecret` to the name of the secret created.
4. Set `ingress.enabled` to `true`.
1. Optionally, to provision the network immediately, set `ingress.router.enabled` to `true`.
2. Optionally, to provision 1 network per namespace, set `ingress.namespacedNetworks` to `true`.
5. Run `helm install` or `helm upgrade`.
Minimum values.yaml example:
```yaml
netbirdAPI:
key: "nbp_XXxxxxxxXXXXxxxxXXXXXxxx"
ingress:
enabled: true
cluster:
name: kubernetes
```
> Learn more about the values.yaml options [here](../helm/kubernetes-operator/values.yaml).
### Exposing Kubernetes API
1. Ensure Ingress functionality is enabled.
2. Set `ingress.kubernetesAPI.enabled` to true.
3. Set `ingress.kubernetesAPI.groups` to a list of groups to assign to the Network Resource to be created for kubernetes API.
4. Set `ingress.kubernetesAPI.policies` to a list of policy names to connect to the resource (See #managing-policies for more details).
5. Apply Helm changes through `helm upgrade`.
6. Replace the server URL in your kubeconfig file with `https://kubernetes.default.<cluster DNS>` (by default `https://kubernetes.default.svc.cluster.local`), for example:
```yaml
apiVersion: v1
clusters:
- cluster:
certificate-authority: /home/user/.minikube/ca.crt
server: https://kubernetes.default.svc.cluster.local
name: minikube
```
### Exposing a Service
> [!IMPORTANT]
> Ingress DNS Resolution requires DNS Wildcard Routing to be enabled and at least one DNS Nameserver configured for clients.
|Annotation|Description|Default|Valid Values|
|---|---|---|---|
|`netbird.io/expose`| Expose service using NetBird Network Resource ||(`null`, `true`)|
|`netbird.io/groups`| Comma-separated list of group names to assign to Network Resource |`{ClusterName}-{Namespace}-{Service}`|Any comma-separated list of strings.|
|`netbird.io/resource-name`| Network Resource name |`{Namespace}-{Service}`|Any valid network resource name, make sure they're unique!|
|`netbird.io/policy`| Name(s) of NBPolicy to propagate service ports as destination. ||Comma-separated list of names of any NBPolicy resource|
|`netbird.io/policy-ports`| Narrow down exposed ports in a policy. Leave empty for all ports. ||Comma-separated integer list, integers must be between 0-65535|
|`netbird.io/policy-protocol`| Narrow down protocol for use in a policy. Leave empty for all protocols. ||(`tcp`,`udp`)|
|`netbird.io/policy-source-groups`| Specify source groups for auto-generated policies. Required for auto-generating policies||Any comma-separated list of strings.|
|`netbird.io/policy-name`| Specify human-friendly names for auto-generated policies. ||comma-separated list of `policy:friendly-name`, where policy is the name of the kubernetes object.|
Example service:
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
name: nginx
namespace: default
labels:
app: nginx
spec:
replicas: 0
replicas: 1
selector:
matchLabels:
app: nginx
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:latest
ports:
- containerPort: 80
- image: nginx
imagePullPolicy: Always
name: nginx
---
apiVersion: v1
kind: Service
metadata:
name: nginx-service
annotations:
netbird.io/expose: "true"
netbird.io/groups: "groupA,groupB"
name: nginx
namespace: default
labels:
app: nginx
spec:
type: ClusterIP
ports:
- name: http
port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
ports:
- protocol: TCP
port: 8080
targetPort: 80
type: ClusterIP
---
apiVersion: netbird.io/v1alpha1
kind: NetworkResource
metadata:
name: nginx
namespace: default
spec:
networkRouterRef:
name: prod
namespace: netbird
serviceRef:
name: nginx
groups:
- name: All
```
### Notes
* `netbird.io/expose` will interpret any string as a `true` value; the only `false` value is `null`.
* The operator does **not** handle duplicate resource names within the same network, it is up to you to ensure resource names are unique within the same network.
* While the NetBird console will allow group names to contain commas, this is not allowed in `netbird.io/groups` annotation as commas are used as separators.
* If a group already exists on NetBird console with the same name, NetBird Operator will use that group ID instead of creating a new group.
* NetBird Operator will attempt to clean up any resources created, including groups created for resources.
* If a group is used by resources that the operator cannot clean up, the operator will eventually ignore the group in NetBird.
* It's recommended that unique groups be used per NetBird Operator installation to remove any possible conflicts.
* The Operator does not validate service annotations on updates, as this may cause unnecessary overhead on any Service update.
### Managing Policies
Members of the `All` NetBird group can now reach the nginx service at `nginx.default.prod.company.internal` through the NetBird network.
Policies can be either created through the Helm chart or they can be auto-generated from Service annotation definitions.
## Client Sidecar
#### Helm
In certain situations we may want to have a pod act like a peer in the Netbird network instead of exposing it through a routing peer. In these cases a Netbird client container has to be added as a sidecar to the pod.
Sidecars are appended to pods when created if they match the selector of a sidecar profile. A sidecar profile defines the configuration of the sidecar, like the setup key to be used along with other parameters. An empty selector will match with all pods in the namespace. The sidecar profile needs to be created first before any pod is created.
Simply add policies under `ingress.policies`, for example:
1. Add the following configuration in your `values.yaml` file.
```yaml
ingress:
policies:
default:
name: Kubernetes Default Policy # Required, name of policy in NetBird console
description: Default # Optional
sourceGroups: # Required, name of groups to assign as source in Policy.
- All
ports: # Optional, resources annotated 'netbird.io/policy=default' will append to this.
- 443
protocols: # Optional, restricts protocols allowed to resources, defaults to ['tcp', 'udp'].
- tcp
- udp
bidirectional: true # Optional, defaults to true
apiVersion: netbird.io/v1alpha1
kind: SetupKey
metadata:
name: sidecar
namespace: default
spec:
name: sidecar
ephemeral: true
---
apiVersion: netbird.io/v1alpha1
kind: SidecarProfile
metadata:
name: test
namespace: default
spec:
setupKeyRef:
name: sidecar
podSelector:
matchLabels:
app: ubuntu
```
2. Reference policies in Services using `netbird.io/policy=default,otherpolicy,...`, this will add relevant ports and destination groups to policies.
3. (Optional) Limit specific ports in exposed service by adding `netbird.io/policy-ports=443`.
4. (Optional) Limit specific protocol in exposed service by adding `netbird.io/policy-protocol=tcp`.
#### Auto-Generated Policies
When a pod matching the selector is created it will receive a netbird sidecar container.
1. Ensure `ingress.allowAutomaticPolicyCreation` is set to true in the Helm chart and apply.
2. Annotate a service with `netbird.io/policy` with the name of the policy as a kubernetes object, for example `netbird.io/policy: default`. This will create an NBPolicy with the name `default-<Service Namespace>-<Service Name>`.
3. Annotate the same service with `netbird.io/policy-source-groups` with a comma-separated list of group names to allow as a source, for example `netbird.io/policy-source-groups: dev`.
4. (Optional) Annotate the service with `netbird.io/policy-name` for a human-friendly name, for example `netbird.io/policy-name: "default:Default policy for kubernetes cluster"`.
```yaml
apiVersion: v1
kind: Pod
metadata:
name: ubuntu
namespace: default
labels:
app: ubuntu
spec:
containers:
- name: ubuntu
image: ubuntu:latest
command: ["sleep", "infinity"]
```
#### Notes on Policies
* Each NBPolicy will only create policies in the NetBird console when the information provided is enough to create one. If no services act as a destination or specified services do not conform to the protocol(s) defined, the policy will not be created.
* Each NBPolicy will create one policy in the NetBird console per protocol specified as long as the protocol has destinations; this ensures better-secured policies by separating ports for TCP and UDP.
* Policies currently do not support ICMP protocol, as ICMP is not supported in Kubernetes services, and there are [no current plans to support it](https://discuss.kubernetes.io/t/icmp-support-for-kubernetes-service/21738).
* NetBird currently does not support SCTP protocol.
Once both containers have started the pod should show up like a peer in the Netbird dashboard.