A real time drum-wise volume visualization system for learning volume-balanced drum performance
108 Views
November 03, 21
スライド概要
To improve drum performance, it is important to consider the volume balance of the bass drum, snare drum, and hi-hat. However, it is difficult for players to evaluate the balance of each volume of these instruments while playing. In addition, while it is possible to self-diagnose by recording drum performances, it is not always efficient to re-record and re-play drum performances based on the correction points. Therefore, we developed a system that uses semi-supervised non-negative matrix factorization (SSNMF) to separate a player’s drum performance, recorded with a unidirectional microphone installed in front of the drum kit, into the bass drum, snare drum, and hi-hat sound sources in real-time, and estimates each volume at the time of beating. In addition, the system also visualizes their volume balance and enables the player to control the power of beating. We experimented using this system for actual drum performance and clarified its usefulness and points for improvement based on the feedback obtained from the experiment participants.
明治大学 総合数理学部 先端メディアサイエンス学科 中村聡史研究室
関連スライド
各ページのテキスト
A Real-time Drum-wise Volume Visualization System for Learning Volume-Balanced Drum Performance Mitsuki Hosoya（Meiji University） Satoshi Nakamura, Masanori Morise（Meiji University） Kazuyoshi Yoshii（Kyoto University）
Overview To support the practice of volume balance of drums, we propose a system that estimates the volume balance from live drum performances and visualizes it in real time.
Background Drums express rhythm by playing multiple instruments simultaneously.
Background To improve one’s drumming, it is important to consider the volume balance of drums of the three main instruments: Hi-hat （HI） Bass drum （BD） Snare drum （SD）
Background Look at 3 videos I played with different volume balances.
Background How does the volume balance of each drum performance sound?
Background The volume balance of HI, SD, and BD for each drum performance is as follows. Hi-hat（HI） Hi-hat（HI） Hi-hat（HI） Soft Loud Soft Snare drum（SD） Snare drum（SD） Snare drum（SD） Medium Medium Medium Bass drum（BD） Bass drum（BD） Bass drum（BD） Loud Loud Soft
Background If the volume of the bass drum is very soft, it may sound like a performance with no stability. Hi-hat（HI） Hi-hat（HI） Hi-hat（HI） Soft Loud Soft Snare drum（SD） Snare drum（SD） Snare drum（SD） Medium Medium Medium Bass drum（BD） Bass drum（BD） Bass drum（BD） Soft Loud Loud no stability no intonation coherent
Background If the volume of the hi-hat is very loud, it may sound like a performance with no intonation. Hi-hat（HI） Hi-hat（HI） Hi-hat（HI） Soft Loud Soft Snare drum（SD） Snare drum（SD） Snare drum（SD） Medium Medium Medium Bass drum（BD） Bass drum（BD） Bass drum（BD） Soft Loud Loud no stability no intonation coherent
Background If the volume decreases in the order of BD, SD, HI... It sounds like a coherent performance. Hi-hat（HI） Hi-hat（HI） Hi-hat（HI） Soft Loud Soft Snare drum（SD） Snare drum（SD） Snare drum（SD） Medium Medium Medium Bass drum（BD） Bass drum（BD） Bass drum（BD） Soft Loud Loud no stability no intonation coherent
Background The audience will sound differently depending on the volume balance of HI, SD and BD. Hi-hat（HI） Hi-hat（HI） Hi-hat（HI） Soft Loud Soft Snare drum（SD） Snare drum（SD） Snare drum（SD） Medium Medium Medium Bass drum（BD） Bass drum（BD） Bass drum（BD） Soft Loud Loud no stability no intonation coherent
Background There are various the ideal volume balances that drummers aim for. →The ideal volume balance is varies depending on the song, genre, and preference. Hi-hat （HI） Medium Hi-hat （HI） Medium Snare drum （SD） Loud Snare drum （SD） Soft Bass drum （BD） Loud Bass drum （BD） Soft Rock/Metal Drummer Jazz Drummer
Background Drummers want to master the ideal volume balance for each and practice it. Hi-hat （HI） Soft Snare drum （SD） Loud Bass drum （BD） Medium Hi-hat （HI） Medium Snare drum （SD） Loud Bass drum （BD） Loud
Background When practicing the volume balance of drums, drummers who attend drum schools can have their drums listened to objectively by lecture. You might want to turn down the volume on the hi-hat. Drummers practicing alone find it difficult to know their own volume balance. How does audience hear the volume balance?
Background The reason why it’s difficult to practice volume balance oneself, there is a gap of perceived volume balance between a drummer and audiences. Power ratio Balance of force to each instrument when the drummer plays the drums. Audience Drummer Perception Volume balance perceived by the drummer while playing. Perception The volume balance that the audience judges when they listen to a drum performance
Background The reason why it’s difficult to practice volume balance oneself, there is a gap of perceived volume balance between a drummer and audiences. Power ratio Balance of force to each instrument when the drummer plays the drums. Drummer Perception Volume balance perceived by the drummer while playing. Gap Audience Perception The volume balance that the audience judges when they listen to a drum performance
Background The reason why it’s difficult to practice volume balance oneself, there is a gap of perceived volume balance between a drummer and audiences. Power ratio Balance of force to each instrument when the drummer plays the drums. One of the problems in improving drumming is that it is difficult to practice volume balance on your own. Drummer Perception Volume balance perceived by the drummer while playing. Gap Audience Perception The volume balance that the audience judges when they listen to a drum performance
Background One of the existing methods for practicing on your own is to listen to a recording of yourself playing, but this method is time-consuming and labor-intensive.
Background In this study, we focused on the problem that when practicing basic rhythms oneself on acoustic drums, • drummers do not know the volume balance perceived by the audience during the performance • it takes time and effort to check the volume balance. What is the volume balance of my drumming? It takes time and effort.
Purpose To support drummers learn the ideal volume balance more quickly and efficiently. What is the volume balance of my drumming? The Hi-hat of my drumming is loud!! system
Proposed System The system that estimates the volume balance of drums and visualizes the estimated volume balance in real-time and feeds it back to the drummer.
Proposed System Recording drum performance Sound-SourceSeparation A Separating three instrument(HI/SD/BD) by Sound Source Separation physical volume HI:0.0012 SD:0.0035 BD:0.0044 estimates psychological volume HI: Medium SD: Loud BD: Soft Estimating the volume balance Visualizing and presenting the volume balance.
Proposed System Recording drum performance Real-time input (recording) of performances from microphones installed at a distance from the drums.
Proposed System Separating three instrument(HI/SD/BD) by Sound Source Separation Separate the input drum performance into three instruments (HI/SD/BD) by sound source separation. Sound-SourceSeparation HI A SD Drum Performance BD
Proposed System Estimating the volume balance Estimate the volume balance perceived by the audience from the combination of the physical quantities(RMS value etc.) of the separated sound sources of each instrument. physical volume HI:0.0012 SD:0.0035 BD:0.0044 estimates psychological volume HI: Medium SD: Loud BD: Soft
Proposed System Visualizing and presenting the volume balance. Visualize and present the estimated volume balance.
Proposed System Our system helps drummers learn the volume balance, as they can learn it quickly by playing while checking the volume balance perceived by the audience. Hi-hat is loud. My drums sound like a loud hi-hat to the audience!
Video of the system in use ビデオ
Implementation The system was operated on a PC, Can change tempo and the target volume balance. The target volume balance setting var The tempo (BPM) setting knob
Implementation Drummer can freely set the volume balance he want to master.
Experimental Overview We conducted an experiment using the system to practice volume balance in drumming to identify improvements in the system.
Experimental Overview The experiment participants were 4 males in their 20s who had been playing the drums for more than 5 years and wanted to improve their volume balance.
Experimental Overview After completion of the task (drumming using system), we administered a questionnaire about proposed system to the participants and interviewed them. Q1 How satisfied were you with the system? 5-pt evaluation （–2 to 2） Q2 Would you want to use the system in the future? 5-pt evaluation （–2 to 2） Q3 Were you able to perform at the set volume balance? 5-pt evaluation （–2 to 2） Q4 What are your thoughts on using the system? Free answer
Results: Q1,Q2,Q3 The average of the evaluation values from Q1 to Q3 were all positive. This results indicated that the user’s impression of the system was not bad. Q1 How satisfied were you with the system? Q2 Would you want to use the system in the future? Q3 Were you able to perform at the set volume balance? 0.75 1 1.25
Results: Q4” Thoughts on using the system” An overall favorable opinion of the system was obtained in Q4, we considered the proposed method can support the acquisition of volume balance in drum performance. Can check volume balance with objective numbers. check my volume balance with concrete figures and graphs. The change in volume was reflected in the system.
Results : Q4” Thoughts on using the system” However, as indicated by the feedback we received in Q4, several points about the system need to be improved. Not linked when I made detailed changes. Difficult to understand how to visualize the results. Want to use it while playing in a band (with other instruments).
Future Work In future work, we will improve the system based on the feedback obtained from this experiment.
Future Work The volume balance estimation was not sufficient. This is because the tuning to correspond the physical volume calculated by the source separation, and the psychological volume judged by the audience was done based on the subjective judgment of the first author. correspond physical volume HI:0.0012 SD:0.0035 BD:0.0044 psychological volume HI: Medium SD: Loud BD: Soft
Future Work The volume balance estimation was not sufficient. →We will experiments in which we ask people to judge the volume balance after listening to drum performances with various volume balances, and to review the method of estimating the volume balance perceived by the audience. HI: Medium SD: Loud BD: Soft
Future Work By improving the system, we plan to make it easier and more efficient to learn how to balance the volume of a drum performance.
Summary Background: It is difficult to practice volume balance in drum performance on your own. Purpose: Helping drummer learn to the volume valance of drumming. Proposed Method: The system that estimates the volume balance of drums and visualizes the estimated volume balance in real-time and feeds it back to the drummer. Future Work: We will improve the system based on the feedback. -Improving the accuracy of the volume balance estimation method. -Improvement of visualization methods and UI.