Encoding an SVG as a data URI lets you embed it directly in CSS or HTML, removing one network request. That is a real win for small icons on a critical path. It comes at a cost: Base64 inflates the data by about a third, and an inlined image cannot be cached separately from the file containing it.
For small icons under roughly 2 KB that appear above the fold, where saving a request matters more than caching. For anything larger, a separate file that the browser can cache independently is usually the better choice.
Base64 represents three bytes using four characters, so the encoded form is about 33 percent larger by design. Minify the SVG first to offset some of that.
Often yes. SVG is text, so it can be URL-encoded rather than Base64-encoded, which avoids the 33 percent penalty entirely and stays readable. Base64 is the safer default when you want to avoid escaping issues.
Separate caching. A linked SVG is downloaded once and reused across every page that references it, while an inlined one is re-sent inside every file that contains it. You also cannot style an inlined data URI with CSS the way you can with inline SVG markup.