cvcg_utils.render package

cvcg_utils.render.camera module

class cvcg_utils.render.camera.BatchDRTKCamera(*args: Any, **kwargs: Any)

Bases: Module

property batch_size: int

proj_points_to_drtk_screen(pts: torch.Tensor, detach_z: bool)

pts: [B, N, 3] or [N, 3]

out: [B, N, 3], batched DRTK screen space coordinates, (-0.5, -0.5) to (W-0.5, H-0.5)

class cvcg_utils.render.camera.DRTKCamera(*args: Any, **kwargs: Any)

Bases: Module

get_campos_no_grad()

proj_points_to_camera(pts: torch.Tensor)

pts: [B, N, 3]

out: camera space coordinates

proj_points_to_drtk_screen(pts: torch.Tensor, detach_z: bool)

pts: [B, N, 3]

out: DRTK screen space coordinates, (-0.5, -0.5) to (W-0.5, H-0.5)

proj_points_to_grid_sample(pts: torch.Tensor, detach_z: bool)

pts: [B, N, 3]

out: torch.nn.functional.grid_sample compatible normalized coordinates, (-1, -1) to (1, 1), use align_corners=False

transform_vectors_to_camera_frame(vecs: torch.Tensor)

pts: [B, N, 3]

unproj_depth_image(depth_img: torch.Tensor)

depth_img: [B, H, W]

out_img: [B, 3, H, W]

class cvcg_utils.render.camera.DiffDRTKCamera(*args: Any, **kwargs: Any)

Bases: Module

clamp_internal_vals_()

focal_0 = 400.0

get_campos_no_grad()

get_focal_cv()

get_focal_cv_no_grad()

get_opencv_matrices_no_grad()

get_projection_matrices()

proj_points_to_drtk_screen(pts: torch.Tensor, detach_z: bool)

pts: [B, N, 3]

out: DRTK screen space coordinates, (-0.5, -0.5) to (W-0.5, H-0.5)

class cvcg_utils.render.camera.GSCamera(tanfov_x: float, tanfov_y: float, world_view_transform: torch.Tensor, projection_matrix: torch.Tensor, full_proj_transform: torch.Tensor, campos: torch.Tensor, H: int, W: int)

Bases: object

camera frame uses the OpenGL convention

the z clip coord is slightly different from OpenGL

classmethod from_intr_extr_torch(intr_3x3: torch.Tensor, extr_4x4: torch.Tensor, H: int, W: int)

cvcg_utils.render.camera.K_from_fov(fov_x, fov_y, fov_mode, H, W)

class cvcg_utils.render.camera.NvdiffrecmcCamera(*args: Any, **kwargs: Any)

Bases: Module

class cvcg_utils.render.camera.UnifiedCamera(K: List[List[float]] | ndarray, R: List[List[float]] | ndarray, T: List[List[float]] | ndarray, H: int, W: int, name: str = None)

Bases: object

H: int

K: ndarray

R: ndarray

T: ndarray

W: int

property campos

crop(x_crop_low: int, y_crop_low: int, new_H: int, new_W: int)

classmethod from_4d_dress(intr_mat_3x3, extr_mat_3x4, h, w, name=None) → Self

classmethod from_actorshq(w, h, rx, ry, rz, tx, ty, tz, fx, fy, px, py, name=None) → Self

classmethod from_avatarrex(cam_dict, name=None) → Self

classmethod from_intr_extr(intr_mat_3x3, extr_mat_3x4, h, w, name=None) → Self

both intr 3x3 and extr 3x4 (w2c) follow the opencv convention

classmethod from_lookat(campos: List[float] | ndarray, lookat: List[float] | ndarray, up: List[float] | ndarray, H: int, W: int, fov_x: float = None, fov_y: float = None, fov_mode: str = None, K: List[List[float]] | ndarray = None, name: str = None) → Self

Provide either (fov_x, fov_y) or K

OpenCV convention w2c matrix priority: front > up > right

classmethod from_yaw_pitch_radius_fov(yaw: float, pitch: float, radius: float, fov_x: float, fov_y: float, H: int, W: int, angle_mode: str) → Self

generates a camera looking at the origin whose position is defined by yaw, pitch and radius

+z is up

property front

make_screen_coords(homogeneous)

name: str

proj_points_to_camera_space(pts: ndarray)

pts: [N, 3]

proj_points_to_screen(points: ndarray)

points: [N, 3]

scale(s_x, s_y)

to_3dgs_format(znear=0.001, zfar=100) → GSCamera

to_drtk_format(znear=0.001, zfar=100) → DRTKCamera

to_idr_format()

to_nvdiffrecmc_format(znear=0.001, zfar=100) → NvdiffrecmcCamera

unproject_depth(depth_img, target_system)

depth_img: [H, W]

cvcg_utils.render.camera.assert_floats(*args)

cvcg_utils.render.camera.assert_ints(*args)

cvcg_utils.render.camera.focal2fov(focal, pixels)

cvcg_utils.render.camera.get_projection_matrix_3dgs(znear, zfar, fov_x=None, fov_y=None, K=None, img_h=None, img_w=None, array_package=<module 'numpy' from '/home/docs/checkouts/readthedocs.org/user_builds/cvcg-utils/envs/latest/lib/python3.13/site-packages/numpy/__init__.py'>, device=None)

cvcg_utils.render.camera.opencv_extrinsics_to_opengl_world2view(R, T)

cvcg_utils.render.camera.opencv_intrinsics_to_opengl_projection(znear, zfar, K=None, img_h=None, img_w=None)

cvcg_utils.render.drtk_renderer module

cvcg_utils.render.drtk_renderer.render_drtk_depth(camera: DRTKCamera | BatchDRTKCamera, verts: torch.Tensor, faces: torch.Tensor, make_differentiable=False)

only renders depth

supports two batching modes: camera batching and vert batching

the function uses isinstance(camera, DRTKCamera) to determine which batching type to use

only one of camera and verts is allowed to be batched

camera: unbatched type DRTKCamera or batched type BatchDRTKCamera verts: unbatched [Nv, 3], or batched [B, Nv, 3] faces: unbatched [Nf, 3], shared by items in the vertex batch bg_attr: [C], shared by items in the vertex batch

cvcg_utils.render.drtk_renderer.render_drtk_face_attr(camera: DRTKCamera | BatchDRTKCamera, verts: torch.Tensor, faces: torch.Tensor, face_attrs: torch.Tensor, make_differentiable=False, allow_neg_depth: bool = False)

supports two batching modes: camera batching and vert batching

the function uses isinstance(camera, DRTKCamera) to determine which batching type to use

only one of camera and verts is allowed to be batched

camera: unbatched type DRTKCamera or batched type BatchDRTKCamera verts: unbatched [Nv, 3], or batched [B, Nv, 3] faces: unbatched [Nf, 3], shared by items in the vertex batch face_attrs: unbatched [Nf, C], shared by items in the vertex batch bg_attr: [C], shared by items in the vertex batch

cvcg_utils.render.drtk_renderer.render_drtk_point_sprites(camera: DRTKCamera | BatchDRTKCamera | DiffDRTKCamera, verts: torch.Tensor, vert_attrs: torch.Tensor, point_size: float, make_differentiable: bool, allow_neg_depth: bool = False)

cvcg_utils.render.drtk_renderer.render_drtk_shaded(camera: DRTKCamera | BatchDRTKCamera, verts: torch.Tensor, faces: torch.Tensor, shading_mode: str, shading_func: Callable, make_differentiable: bool = False, allow_neg_depth: bool = False)

verts: batched [B, Nv, 3] faces: unbatched [Nf, 3], shared by items in the vertex batch

shading_func: a callable that turns normals […, 3] to rgb […, 3]

face_attrs: unbatched [Nf, C], shared by items in the vertex batch bg_attr: [C], shared by items in the vertex batch

cvcg_utils.render.drtk_renderer.render_drtk_uv_textured(camera: DRTKCamera | BatchDRTKCamera, verts: torch.Tensor, faces: torch.Tensor, uv_verts: torch.Tensor, uv_faces: torch.Tensor, texture_img: torch.Tensor, make_differentiable: bool = False, flip_v: bool = True, allow_neg_depth: bool = False)

verts: batched [B, Nv, 3] faces: unbatched [Nf, 3], shared by items in the vertex batch uv_verts: unbatched [Nv, 3] uv_faces: unbatched [Nf, 3], shared by items in the vertex batch texture_img: [B, C, H, W]

out: textured_render_img: [B, C, H, W] mask: [B, H, W] face_index_img: …

cvcg_utils.render.drtk_renderer.render_drtk_vert_attr(camera: DRTKCamera | BatchDRTKCamera, verts: torch.Tensor, faces: torch.Tensor, vert_attrs: torch.Tensor, attr_faces: torch.Tensor, make_differentiable: bool = False, allow_neg_depth: bool = False)

similar to render_drtk_face_attr, supports two batching modes

whether vert_attrs is batched should be consistent with camera and verts

attr_faces: unbatched [Nf, 3]

if verts and vert_attrs align, then attr_faces == faces, but distinguishing them allows other cases, e.g., uvs

cvcg_utils.render.dtgs_renderer module

cvcg_utils.render.gs_renderer module

cvcg_utils.render.gs_renderer.make_gs_rasterizer(camera: GSCamera, bg_color, scaling_modifier=1.0, active_sh_degree=3, debug=False, antialiasing=False)

cvcg_utils.render.gs_renderer.quat_from_axis_angle(npts, axis, angle)

cvcg_utils.render.gs_renderer.render_gs(camera: GSCamera, xyz: torch.Tensor, opacity: torch.Tensor, scales: torch.Tensor, rotations: torch.Tensor, features: torch.Tensor, active_sh_degree: int, cov3D_precomp: torch.Tensor = None, override_color=None, compute_cov3D_python: bool = False, convert_SHs_python: bool = False, use_trained_exp=False, clip_value=False)

Render the scene.

Background tensor (bg_color) must be on GPU!

xyz: [N, 3] opacity: [N, 1] scales: [N, 3], overridable by cov3D_precomp rotations: [N, 3], overridable by cov3D_precomp cov3D_precomp: [N, 6], stores only the lower diag. use strip_symmetric to obtain this) override_color: [N, 3] …

cvcg_utils.render.gs_renderer.strip_lowerdiag(L)

cvcg_utils.render.gs_renderer.strip_symmetric(sym)

cvcg_utils.render.hybrid_drtk_gs_renderer module

cvcg_utils.render.hybrid_drtk_gs_renderer.render_gs_and_drtk_uv_textured(uni_camera: UnifiedCamera, gs_xyz: torch.Tensor, gs_opa: torch.Tensor, gs_cov: torch.Tensor, gs_rgb: torch.Tensor, gs_lab: torch.Tensor, m_camera: DRTKCamera, m_verts: torch.Tensor, m_faces: torch.Tensor, m_uv_verts: torch.Tensor, m_uv_faces: torch.Tensor, m_texture_img: torch.Tensor, m_label_img: torch.Tensor, m_make_differentiable: bool = False, m_detach_z: bool = False, m_flip_v: bool = True)

for compatibility, both GS and DRTK use unbatched data here

verts: batched [Nv, 3] faces: unbatched [Nf, 3], shared by items in the vertex batch uv_verts: unbatched [Nv, 3] uv_faces: unbatched [Nf, 3], shared by items in the vertex batch texture_img: [C, H, W]

face_attrs: unbatched [Nf, C], shared by items in the vertex batch bg_attr: [C], shared by items in the vertex batch

cvcg_utils.render.sh_utils module

cvcg_utils.render.sh_utils.RGB2SH(rgb)

cvcg_utils.render.sh_utils.SH2RGB(sh)

cvcg_utils.render.sh_utils.eval_sh(deg, sh, dirs)

Evaluate spherical harmonics at unit directions using hardcoded SH polynomials. Works with torch/np/jnp. … Can be 0 or more batch dimensions. :param deg: int SH deg. Currently, 0-3 supported :param sh: jnp.ndarray SH coeffs […, C, (deg + 1) ** 2] :param dirs: jnp.ndarray unit directions […, 3]

Returns:

[…, C]

cvcg_utils.render.sh_utils.rotate_sh(shs, rotmat)

Module contents