Background: Epitranscriptomics is a relatively new field of study focused on post-transcriptional regulation of gene expression and encompasses modifications found on a variety of RNA molecules, including rRNA, tRNA, and mRNA. Advances in RNA modification detection methods require novel analysis techniques and tools. Recently developed direct RNA sequencing approaches involve complex ionic current data and numerous analysis techniques, thus prompting a thorough assessment of the strengths and limitations of using the latest RNA modification detection tools with direct RNA sequencing data from biological samples. Methods: We sequenced the RNA from a variety of sources including bacteria, fungi, animals, and viruses with Oxford Nanopore Technologies direct RNA sequencing. To examine the presence of RNA modifications, we used available analysis software and developed our own modification prediction tool as well. To interrogate the effects of an RNA modification enzyme on a whole eukaryotic transcriptome, we used CRISPR-Cas9 to generate a mutant Drosophila melanogaster fly line with a deletion in the cytosine methyltransferase Mt2, followed by modification detection with the latest basecalling models developed by Oxford Nanopore Technologies. Results: We found that biases towards specific sequences and failure to account for differences in sequencing depth contribute to the low accuracy of many of the current RNA modification detection tools. We developed a tool, DRAMA, to overcome some of the limitations to modification detection by normalizing ionic current statistical data based on the sequencing depth. In experiments focused on the most common modifications found in RNA, modification basecalling models using the Dorado software provide a relatively sensitive and specific option. Applying this approach in a differential modification analysis using a D. melanogaster methyltransferase knockout revealed a complex system of epitranscriptomic regulation throughout the transcriptome. Conclusion: There are numerous barriers to accurate RNA modification detection including low accuracy of commonly used analysis software. Interrogation of RNA modification dynamics may be improved through the use of multiple tools for higher confidence predictions, and the strengths and limitations of each method should be considered. A comparative approach, such as the use of a methyltransferase knockout, can provide insight into epitranscriptomic regulation within a biological system.