Advancing AI sports analytics through combined power of a synthetic data-driven SKY ENGINE AI platform and NVIDIA RTX

Team-based sports — an attractive opportunity for Machine Learning and Computer Vision

Main advantages of our approach include:

  • Efficient dealing with unbalanced data
  • Accurate detection of logotypes on uniforms and stadium
  • Precise labells and 3D ground truth can be generated automatically
  • AI models for human detection and pose estimation can be trained to operate in a specific, unusual environments with just a few lines of code
  • Usually noisy, low quality data stream with compression artifacts does not significantly deteriorate the AI-driven inference accuracy
  • Unknown parameters of broadcast cameras can be effectively derived
  • High quality of 3D mapping available
  • 3D pose estimation for small objects can be accurately carried out
  • Complex structures of movements and formations can be accurately recognized
  • Very efficient data processing and computation optimization with RTX architecture

Sport analytics case with SKY ENGINE AI platform

renderer_ctx.load_abc_scene('stadium')
renderer_ctx.setup()
with example_assistant.get_visualizer() as visualizer:
visualizer(renderer_ctx.render_to_numpy())
Figure 1. Preview of the 3D stadium geometry in the SKY ENGINE AI platform. A simplified preview of stadium geometry with simple Phong shader, without materials. Grayscale image.
stadium_base_textures = SubstanceTextureProvider(renderer_ctx, 'concrete')
stadium_base_params = PBRShader.create_parameter_provider(renderer_ctx, tex_scale=50)
renderer_ctx.set_material_definition('stadion_base_GEO', MaterialDefinition(stadium_base_textures, parameter_set=stadium_base_params))
renderer_ctx.define_env(Background(renderer_ctx,
EnvMapMiss(renderer_ctx),
HdrTextureProvider(renderer_ctx, 'light_sky')))
renderer_ctx.layout().duplicate_subtree(renderer_ctx, 'player_GEO_NUL', suffix='team2')
renderer_ctx.layout().get_node('player_GEO_NUL').n_instances = 20
renderer_ctx.layout().get_node('player_GEO_NUL_team2').n_instances = 20
shirt_sync = SynchronizedInput(SynchronizationDescription(in_strategy=Synchronization.DISTINCT_EQUAL_GROUPS))
player_material_strategy = DrawingStrategy(renderer_ctx, inputs_strategies={'Colors_select': shirt_sync})
renderer_ctx.instancers['player_GEO'].modify_material_definition(strategy=player_material_strategy)
renderer_ctx.instancers['player_GEO_team2'].modify_material_definition(randomization_group='team2', strategy=player_material_strategy)
player_geometry_strategy = DrawingStrategy(renderer_ctx, frame_numbers_strategy=UniformRandomInput())
renderer_ctx.instancers['player_GEO'].modify_geometry_definition(strategy=player_geometry_strategy)
gauss_strategy = DrawingStrategy(renderer_ctx, default_input_strategy=RandomGaussianRandomInput(sigma_relative_limits=(0.1, 0.2)))
renderer_ctx.layout().get_node('player_GEO_NUL').modify_locus_definition(strategy=gauss_strategy)
example_assistant.visualized_outputs = {SceneOutput.BEAUTY, SceneOutput.SEMANTIC, SceneOutput.KEYPOINTS}
datasource = MultiPurposeRendererDataSource(renderer_context=renderer_ctx, images_number=20, cache_folder_name='rugby_presentation_new')
main_datasource = SEWrapperForDistancePose3D(datasource, imgs_transform=transform)
train_data_loader = DataLoader(dataset,
batch_size=Constants.TRAIN_BATCH_SIZE,
num_workers=Constants.NUM_WORKERS,
drop_last=Constants.DROP_LAST,
shuffle=Constants.VALID_SHUFFLE,
collate_fn=collate_fn)
model = get_pose_3d_model(main_datasource.joint_num, backbone_pretrained=True)
trainer = DefaultTrainer(
data_loader=train_data_loader, model=model, epochs=Constants.EPOCHS, save_freq=1,
valid_data_loader=valid_data_loader, optimizer=optimizer, evaluator=evaluator, scheduler=scheduler, serializer=serializer)
trainer.train()
show_jupyter_picture('gtc03_assets/trained/img2.png')
checkpoint = torch.load('gtc03_assets/trained/rugby_detection.pth.tar')
for k, v in sorted(checkpoint.items()):
  checkpoint[''.join(['_model.', k])] = checkpoint.pop(k)
detection_model.load_state_dict(checkpoint)
detection_model = detection_model.to(device)
real_dataset = ImageInferenceDatasource(dir='gtc03_assets/real_data', extension='png')
out = outputs.pop()
bboxes = out['boxes'].cpu().detach().numpy()
bboxes = bboxes[np.where(labels == 1)[0]]
labels = out['labels'].cpu().detach().numpy()
bbox_image = bboxes_viz(orig_img, bboxes)
model.eval()
with torch.no_grad():
  results = model((img,), ({'boxes': torch.from_numpy(bboxes).int()},))
results = results.pop()
output_coords, output_bboxes = results['pred_poses_coords'].cpu(), \
        results['boxes'].cpu()

Conclusions

SKY ENGINE AI — Evolutionary AI Platform for Deep Learning in Virtual Reality for ANY Computer Vision applications.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Enhancing Customer Experience with Artificial Intelligence

What’s the Deal with AI?

Hey everyone!

AI-based decision-making automation

Human-AI Collaborated Fashion

Transforming the Business Banking Experience with AI

AI redefining Customer Support

Artificial Intelligence (AI) is the ability of a computer program or a machine to think and learn.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
SKY ENGINE AI

SKY ENGINE AI

SKY ENGINE AI — Evolutionary AI Platform for Deep Learning in Virtual Reality for ANY Computer Vision applications.

More from Medium

Try to get your own cutie portrait using MMGEN-FaceStylor

How Intel enhanced photorealism using machine learning techniques

Artificial Intelligence: Week #1 | 2022

Paper Review: Patch-based inference for TinyML